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I propose . . to argue that up to now we have had no explicitly 
stated complete psychological theory, but rather, that we seem to have 
contented ourselves with a collection of discontinuous, unrelated 
“theorettes.” It is my opinion that psychology has now reached. the 
stage where it has accumulated sufficient data and understanding to 
reformulate, explicitly, its entire theoretical framework (p. 66). 


David Krech (1949) 


Pure empiricism is a delusion. A theorylike process is inevitably in- 
volved in drawing boundaries around certain parts of the flux. of 
experience to define observable events and in the selection of the events 
that are observed. Since multitudinous events could be observed and an 
enormous number of relationships could be determined among all of 
these events, gathering all the facts with no bias from theory is utterly: 
impossible. Scientists are forced to make a drastic selection, cither con- 
sciously on the basis of perceptual habits and the folklore and linguistic 
categories of the culture, or consciously on the basis of explicitly form- 
ulated theory (p. 200). 


Neal E. Miiler (1959) 


If any large trends are detectable, they would be the finer and finer 
differentiating of the subject matter falling under the rubric of learning 
anda eradual merger of “learning theories" w ith theories of perception, 
neuro-physiology. motivation, and personality into "behavior theories.” 
One might predict that rather than a third edition of Theories of learn- 
ing, Hilgard might well write a book 10 years hence titled Theories of 
behavior. We shall see (p. 134- 


Edward L. Walker (1957) 
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If the proverbial Man from Mars or, what would amount to much 


an intelligent layman were to examine a cross-sectional 


the same thing, A 
osvchologv. of learning, he 


"Mice" of the technical literature on the p 
Would find it very confusing. If, for example, he contined himself to 
the contents of our specialized journals and monographs for a given 
vear, he could hardly escape the impression that, while extremely vig- 
f ed and disorderly. A colleague has estimated 
ately 80 new titles appear in this field; and 
( a five-day week, there would be 
a day for him to master. But 


orous, the field is fragment 
that cach month approxim 
if our hypothetical analyst worked 
an average of four articles or monographs 
" in this relatively. immense literature, he 


after thus immersing himself 
ng learned very little about 


might still emerge with the fecling of havi 
learning. Even some professional psychologists would agree with an 
x ect that "all this work on learn- 


opinion recently expressed to the etf i | 
ing is a maze of ‘crucial experiments, none of which is ever quite con- 
firmed and any one of which almost certainly has been contradicted!” 


If, however, one takes a sufficiently long view of the situation, move- 


takable and meaningful kind is apparent. It is the pur- 


ment of an unmis : 
rpret this movement. 


pose of this book to trace and inte 

There are, of course. a number of excellent summarizing volumes in 
the area of learning which are 
these attempt to order the field topically, with only incidental atten- 
tion to the historical dimension; others synopsize major theories, with 


organized along other lines. Some of 


Varying interest in the possibility of unifying them. Here we shall be 


concerned both with convent 
approach will be predominantly developmental, in 


onal topics and with salient theoretical 


Positions; but our 
IX 
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it is believed, both em- 
pirical facts and divergent theories become maximally 
most significantly related. 

Our aim then, in short, is to achieve 
"This, of course, is what every theory 


the historical sense of the term. In this w av, 


meaningful and 


a high-level sy thesis of the field. 


of learning has attempted to do. 
“That such efforts have not been entirely 


successful is, of course, at- 
tested by their very diversity. 


So why should we expect vet another 
attempt. to be more successful Two considerations, already alluded 
to, are relevant: (1) the fact that we shall examine both rescarch and 
conjecture in a broadly historical context and (2) the fact that today 
we have access to many new experimental findings not available to 
earlier system makers. ` 

From all this it should not be inferred, however, that we shall here 
discard or disregard earlier theories of learning. Quite the contrary! 
We shall, indeed. use them as the very cornerstones and building blocks 
for our present edifice, but with the privilege of reshapin 
terpreting them, so as to make th 
all system which, hopefully, will have both greater scope and power 
an any of the earlier conceptual schemes, taken separately. Here we 
yall assume, more specifically, that almost all the 
Contemporary theories. of learning 
large clement, of validity 


rand rcin- 
em fit more smoothly into a new, over- 


( 


cA ek 


great traditional and 
have an clement, perhaps a very 
; and our quest will be for that scheme which 
most fully utilizes these theories bur. 


at the same timc, permits us to sec 
the errors and inadequacies of cach. 

Recently à wag observed. that 
thinks that the human race 
From the 
reader 


a psychologist is "an individual who 
is directly: descended. from the white rat." 
extent to which rat experiments are cited in this book, the 
may conclude that this definition is more true than humorous. 
These experiments are cited so extensively, not because our interest is 
primarily in the white rat 
such, but because, 
atic thinkers in th 


(or any other 
over the course of half a century the great, system- 
e field have gravitated toward animal rescarch; and 
the persevering reader will see why this should have be 
However, this is not to say that even those who most assiduously culti- 
vate this approach to know ledge necessarily: believe that it will unlock 
all the Mysteries of human c i 


"Sperience and existence, 
approach—the approach, 


50 to sav, from below: 

votees are modest, though sincere, in the claims they make for ir. With 

Hamlet they would agree, when he says, R 

heaven and earth, Hor 1 
Some re 


infrahuman organism) as 
en the case. 


It is simply one 
and most of its de- 


“There are more things in 
atio, than are dreamt of in Vou 


ir philosophy.” 
aders may deplore the 


absence, at the end of this volume, of 
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a summary chapter. Such a chapter is omitted for a variety of reasons, 
one being that the basic argument is epitomized in Chapter 7. Earlier 
chapters provide the logical and factual background from which this 
argument evolves; and the five subsequent chapters amplify and apply 
a 


the argument in more specific ways. Thus, the reader who wishes 
quick "look" at this volume as a whole may first read the chapter in- 
dicated: but the argument will unfold most naturally and persuasively: 
if the chapters are read in the order in which they appear. 

A second reason why there is no summary chapter at the end of this 
book is that such a chapter appears as the Introduction to a companion 
volume entitled Learning Theory and the Symbolic Processes (John 
Wiley & Sons, Inc.. 1960). Fxperience has shown that the contents of 
) properly supplemented by collateral reading 


the present book, when 
a full semester's work for advanced 


and classroom discussion, makes 
eraduate students. Yet this material does not by any 


knowledge in the field. Instead it provides a 
broad and apparently secure foundation for study of the more intricate 
topics considered in the second book. Because of the close continuity 
c would have been an advantage, for some 


undergraduate or 
Means exhaust available 


between these books, ther } 
ombined contents appear as a single volume. 


for students of the two-volume format 
The device of summarizing 


readers, in having their € 
But the greater convenience 
has seemed to be a weightier consideration. l ; f 
the present volume at the outset of the ensuing one Increases the feasi- 
bility of this arrangement. 


This word, now, of forew i ] - 
When | have used this book (in mimeographed 


a my unvarving experience that for 


arning—and encouragement—to students 


and their teachers. 
form) as a classroom text, it has beer 2 : 
the first third or half of the course, students are not a little bewildered. 
In their own words, they don't see “what it’s all about.” And for ex- 
just have to emo) 

happens! They discover, often 


ane ae M certain phrases : 
aminations they sav they uin phrases and 


facts. But then something exciting. app à 
quite suddenly, that they have command of a conceptual scheme which 
gives order and meaning to all they have routinely learned and which 
also opens up for them new vistas of insight and inquiry. Once this 
point is reached, the battle is, of course, won. And to the end of keeping 


them in the frev up to this poit 


| have introduced what may í c 
"jn Subject Index). As already indicated, the 


at and preventing premature desertions, 
be termed recurrent. progressive sum- 


maries (see "Summaric : è é . 
is nowhere fully. review ed in this volume. But 


argument as a whole i 
id, in effect, ask: Now where are we and where 


repeatedly we stop ar 


do we go from he At some point—usually in Chapter 5 or Chapter 6 
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—the student begins to grasp the logic of the argument in its totality 
and from that point on needs no such aids. Details click into place, and 
he sees the picture whole. Other instructors who employ this book 
may find ways of speeding up this comprehension of the system, but 
it is believed that a certain minimum period of "maturation" is essential 
as a preliminary and cannot be altogether. eliminated. . 
Persons in other professional disciplines are invited to examine this 
and the ensuing volume in the context of their particular interests. In 
a number of related ficlds the psychology of learning is admittedly 
basic, and it is believed that the approach here followed provides a 
maximally satisfactory introduction and guide to the understanding 
thereof. Criticisms and suggestions from readers will be gratefully re- 
ceived by the author and incorporated in later editions, if there is con- 
tinuing demand for this work. Also, readers arc invited to write to the 
author and obtain, without charge, one of the phonograph recordings 
alluded to in Chapter 7. 
To Mrs. Herbert. Goldstein, Miss Anita Jeanne Elder, and Miss 
Leona D. Pedigo, consecutively my secretaries during the five years 
that this book and its sequel have been in preparation and in press 
my sincere thanks for their sustained interest and efficiency. It is like- 
wise a pleasure to acknowledge my indebtedness to seve 
tions” of graduate students for much help 
aration of these two volumes; and to 


TO 


"i. 


“genera- 
and stimulation in the prep- 
Dr. Donald W. Zimmerman 
and Dr. Janusz Revkowski, who thoughtfully: read all of the final ty pe- 
script copy, go my special thanks. But I am most deeply indebted to 
those three seasoned experts on. behavior theory—John P. Seward, 
Richard S. Solomon, and M. E. Bitterman—who read and mercilessly 
criticized the carly, mimeographed version of these two volumes. 
Whatever their remaining defects, these books have been substantially 
improved by the trenchant commentary of these men; and here I ré- 
cord not only my appreciation of their assistance, but also my delight 
in working with them in this venture. 


O. Honanr Mowrer 
Urbana, Illinois 
December, 1955 
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Introduction: Historical Review 
and Per. spectre 


erstand and evaluate contemporary 


Sometimes the best way to und 
them in historical perspective. Such is the method 


ory chapter, 
and technical discussion of the psychology 


events is to examine 
employed in this introduct 
text for the more specific 
of learning which follows. 


a means of providing con- 


l. Biological Concepts and the Birth 

of Scientific Psychology 

In his book. Limitations of Science, J. W. N. Sullivan (1949) quotes 
Lord Kelvin to the cffect that “he could understand nothing of 
which he could not make a working model" (p. 39). As will be- 
come evident in later chapters. psychologists have often been greatly 
influenced by models. and modes of thought taken—sometimes, it 
seems, quite. unconsciously —from i 
(sce Wiener, 1948. ppe 49750: English. 1954. pp- 174: also. Mowrer, 
1960, Chapter 7). But ps) chology has also drawn heavily from and 
been much influenced by developments in the more immediately 


adjacent field of biology- 
At the beginning of the p 

or "eligo; OF psychology 

to be known as Structuralism. 


it—was upon 
non of consciousness, as such" (p. 1). The 


1 


physics and practical mechanics 


resent century, the prevailing conception 
in this country was w hat has since come 
Here the cmphasis—as Ladd, writing 

^ 


in 1902, phrased “the systematic description and ex- 


planation of the phenome 
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mind, or psyche, was seen as an entity or structure, and the task of 
psychologists was to identify its limits and to. relate and classify its 
various sections. and subdivisions, sometimes. called “faculties.” Like 
Gaul, the mind, as conceived by most. Structuralists, was divided 
into three parts: Cognition (know ing ), Conation (Ow ling). and atfection 
(feeling). And textbook writers spent much space arranging and rc- 


arranging various mental activities and capacities. under these three 
salient categories, Labels and definitions were highly important; and 
the whole approach was essentially descriptive, tayonomic, static. In 
fact, the main objective of the enterprise seemed focused on an cf- 
fort to do for psy chology what Karl von Linne, in 735. had done 
for biology in announcing his great classificatory system, beginning 
with phyla and proceeding 
Species. 

But then, in 1859. Charles. Darwin published. his book, Origin 
of the Species, and a revolution shocked and shook biology and related 
h. HW 


t down through classes, orders, ete.. to the 


sciences. Like Linnaeus, Darwin too was interested in species, i 
the problem of ordering and classifvine living organisms; but he took 


the further, momentous step of also asking how they “gor that way.” 
In other words, he was concerned, as Linnaeus had not been, with 
thc problem of origin, or development; and this interest, combined 
with exhaustive, painstakine observation and analysis, led to a dynamic, 
functional conception know n as Organic Evolution, Slight variations 
in structure, reasoned Darwin, would give to certain individual or- 
ganisms, in a particular environment, slight or perhaps crucial ad- 
vantages in the “struggle for existence.” with the result that. the 
“fittest” would survive best and would leave more offspring. than 
those individuals not showing the same "natural," or "spontaneous," 
variation, By this process, or “mechanism,” organic evolution, w hereby 
all present life had developed. from simpler forms, was thought to 
have come about.! 

For biology. this conception of evolution of form, on the basis 
of function, ushered in a new era. Definition, description, classifica- 
ton were no longer all-important, Consuming interest now centered 
Upon a dynamic phenomenon known as adaptation. Behavior and 
Process suddenly took on special significance, and the whole of life 


ip ids 
resent "origi ies” i 
eser thinking about the origin of species" seems to differ from that of 
Darwin chiefly in thar new 


the concept of mutation, 
replaced the notion of 


knowledge from the field of genetics has introduced 
which has importantly supplemented and, in some degree, 


Spontaneous" variations, Otherwise, a century. later, the 
theory stands intact, : 
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became 
repeats phylogeny” became a new and majestic phrase; and if Man 
lost something of the status he had enjoved in the carlier conception 
of Special Creation, all life now gained new importance, direction, 
and meaning in the evolutionary scheme of things. 

This revolution in the field of biology gradually made itself felt 


yrogressive, developmental, forward-looking, “Ontogeny 
prog I S g 


in psychology; and by the turn of the century a new school of 
thought, known as Fuactionalisi, was coming rapidly to the fore. 
In fact, by. 1906, it was so well established and so respectable that one 
of its great leaders, Dr. James Roland. Angell, was in that year 
elected president of the American. Psychological Association. And 
in his presidential address (1907), entitled “The Province of Func- 
tional Psychology.” he gave a masterful summary and appraisal of 
the nature and ‘significance of this new movement. 

At the outset. Angell acknowledged the relationship between 
Functionalism and Darwinian evolution. He said: 

Whatever else it may be, functional psychology is nothing w holly. new. 
In certain of its phases it is plainly discernible in the psychology: of Aristotle 
and in its more modern garb it has been increasingly in evidence since 
Spencer wrote his Psychology and Darw in his Origin of species (p. 62). 

This involves the identification of functional psychology with the effort 


ay the typical operations of consciousness under actual 
: à attempt to analyze and describe its cle- 


to discern and portr 
life conditions, as over against the 
mentary and complex contents (pp. 62-634. 


After referring to the “much-abused faculty psychology,” or Struc- 


turalism, Angell continued: 

Tie nut of this classic target for psychological vituperation recalls 
the fact that when the critics of functionalism wish to be particularly un- 
pleasant, they refer to it as the bastard offspring of the faculty psychology 
masquerading in biological plumage (pp. 63 
ally, were the functionalist’s position and 


-64). 


But what, more specific 
program? Said Angell: 
is wont to take his cue from the basal con- 
ment, ie, that for the most part organic 
structures and functions possess their present characteristics by virtue. of 
the efficieney with which they fit into the extant conditions of life broadly 
designated by the environment. With this conception of mind he proceeds 


to attempt some unders anding of the manner in which the psychical con- 
tributes ro the furtherance of the sum total of organic activities, not alone 


the psychical in its entirety, but especially the psychical in its particularities 


—mind as judging, mind as feeling, ete. (pp- 68-69). , 
This is the point of view which instantly brings the psychologist. cheek 


by jowl with the general biologist (p. 69)- 


The psychologist of this stripe 
ception of the evolutionary move 
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In other words, in this new frame of reference, mind was to be 
viewed as an organ of adaptation and the focus of interest was to be 


upon its activities, rather than upon its structure or contents. 


Such an effort if successful | said Angell] would not only broaden the 
foundations for biological appreciation of the intimate nature of. accom- 
modatory process, it would also immensely enhance the psychologists 
interest in the exact portraval of conscious life. It is of course the latter 
consideration which lends importance to the matter from our point of view. 
Moreover, not a few practical consequences of value may be expected to 
flow from this attempt, if it achieves even a measurable degree of success. 
Pedagogy and mental hygiene both await. the quickening and guiding 
counsel which can only come from a psychology of this stripe. For their 
purpose a strictly structural psychology is as sterile in theory as teachers 
and psychiatrists have found it in practice (p. 69).2 


In a later chapter, we shall have occasion to refer again to 
this important paper by Angell; but if we are to be faithful to history, 
we must now consider another development which was to intervene 
before the objectives of Functionalism were to sec anvthing like ful- 
fillment. If Angell, at the time he delivered his presidential address, 
could have glimpsed the future a decade hence, he would have seen 
the same function being performed, not by a member of his own 
school, but by the leader of this other movement (Watson, 1916b). 


Il. Behaviorism: Revolt and Reformation 


Frequently Behaviorism is dated from the appearance of two books 
by John B. Watson (1914. 1919) during the second decade of the cen- 
tury. But behaviorism (with a little "b7) certainly had its inception 
well before 1900. During the first decade of the century, Structuralists 
and Functionalists alike continued to stress the phenomenon of con- 
sciousness. But as early as 1896, John Dew ev had already voiced a 
growing di satisfaction with this emphasis and had indicated the 


mode of thought that was to go far toward replacing it. Said Dewey: 


, That the greater demand for a unifying principle and controlling work- 
ing hypothesis in psychology should come at just the time when all gen- 
eralizations and classifications are most questioned and questionable is 
natural enough. It is the very culmination of discrete facts creating the 
demand for unification that also breaks down previous lines of classifica- 
tion... . The idea of the reflex are has upon the whole come nearer to 


For another, more recent review and appraisal of the impact of the theory of 


Selo upon American psychology, sce Boring (1950); also Roc & Simpson 
1958). 
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meeting this demand for a general working hypothesis than any other single 
concept. It being admitted that the sensori-motor apparatus represents both 
the unit of nerve structure and the type of nerve function, the image of 
and became an organizing 


this relationship passed over into. psychology 
principle to hold together the multiplicity of fact (p. 375, italics added). 


in his Principles of Psychology (1890). William James 
method of approach to certain psychological topics 


Even earlier, 
had adopted. a 
which was clearly behavioristic, as the concept. was later to develop. 
ysiology and medicine, this writer freely im- 


Vormally trained in. ph r 
es observations and theories which departed 


ported from these disciplin l 
radically from what was then considered psychology. For example, 


in his chapters on “The Functions of the Brain.” "Some General Con- 


ditions of Brain. Activity.” and "Association," J 
have the term for) reflex conditioning. 
In fact, he even went so far as to reproduce Meynert’s earlier (1874) 
"brain-scheme" (really a constellation of reflex arcs) purporting to 
; burnt, will thereafter avoid touching a 


ames showed a good 


grasp of (though he did not 


show how a child, once 
candle. 

Writing retrospectively of this sam 
(1927) was later to remark: 


e decade (1890-1900), Pavlov 


Under the influence of these new tendencies in biology. which appealed 
to the practical bent of the American mind, the amenican € i Psy- 
chologists—already interested in the comparative smdy af pe Ui m 
evinced a disposition to subject the highest nervous rela o s s i 
experimental analysis under various spec ly des jy S et 5 7 pee 
fairly regard the trencise. by xl horndike, | gee m me 1898], a 
the starting point for systematic investigations of pes m % i -— 

At about the same time as T horndike was en up S. S i ae i 
(being then quite ignorant of his r rches) i as a iu calci pase 
study of the hemispheres. » + - As a result of this dn estigatic i qua 
fied conviction of the futility methods of inquiry was firmly 


stamped upon my mind (pp- 


of subjective 
5-6). 


While Behaviorism was not to find its most explicit spokesman for 
another decade, it was clear by the turn of the century that the “study 
of consciousness," whether by the approach of the Structuralists or 
that of the Functionalists, was rapidly giving way to a lively interest 
in something called behavior. Thus, as carly as 1904. Cattell, in speak- 


j i researc » at Columbia University, was able 
ing of psychological research done at Col \ 


to s 


It seems to me that most of the research work that has been done by me 
independent of introspection as work in 


or in my laboratory is nearly as ur 1 
physics or in zoology. The ume of mental processes, the accuracy of per- 
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ception and movement, the range of consciousness, fatigue and | practice, 
the motor accompaniments of thought, memory, the association of ideas, 
the perception of space, color-vision, preferences, judgments, individual 
differences, the behavior of animals and of children, these and other topics 
I have investigated without requiring the slightest introspection on the pare 
of the subject or undertaking such on my own part during the course of 
the experiments. It is usually no more necessary for the subject to be a 
psychologist than it is for the vivisected frog to be a physiologist (p. 180, 
Italics added ). 


Hence, the formal enunciation of the tenets of Behaviorism by 
Watson, between 1910 and 1920. came as something of an anticlimax, 
since both the ideology and many of the experimental techniques 


of this type of psychology were already well established. Yer Watson 
marshaled forces and led the revolt against the older order in a way 
not equaled by any other figure during the period under review." 

As later chapters of this book will show, Behaviorism is today not- 
able, not so much for its own inherent durability, as for the role it 
has played in a highly creative dialectical process. If. Subjectivism 
(or Introspectionism ) was the original theme, or thesis, in American 
psychology (as it became an independent discipline, distinct from 
philosophy and theology ), Behaviorism was the strident antithesis. 
And something which, for want of a better term, may be called neo- 
behaviorism is the present-day outcome, or synthesis. The present 


* Recently. Dr. Max F. Meyer (who, in 1911, published the first book on psy- 
chology with “behavior” in the title) wrote a letter to the author which reads, 
in part, as follows: 

“Skinner says in the last | January, 1959] issue of Science that \Watson—who 
is compared with Darwin, strangely—cleared the way for a scientific analysis, 

“That way was already pretty clear in 1896. In that carlier year, still in the 
Nineteenth century, | attended the International Congress of Psychology in Mu- 
nich. There was a vigorous debate on that same ‘wi 


between Lipps of Munich 
and a Russian psychologist of the name of Bekhterev 


© The question was simply 
which way should be preferred by university professors. But it was certainly not 
‘unclear’ in the Nineteenth century. That is, on the continent of Furope. 

“What one can truly say of Watson is: He knew how to advertise that ^w ay” 
among English-speaking psychologists.” 


And Woodworth (1958) makes the same point when he says: “Even before the 
advent of behaviorism about 1912, psychologists were beginning to regard the 
word behavior as a good term for the subject matter of their science, a better 


(p. 


term for the purpose than 
20). For example, 
psychology: ^P. 


consciousness or mind or even mental proc 
“Wiliam McDougall in 1905 had offered a new definition of 


| sychology may be best and most comprehensively defined as the 
positive science . .. of the 


tions with other creatures an 
gists offered simil 


activities by which any creature maintains its 
d with the world of physical things. Other psycholo- 
ar comprehensive definitions" (p. 21). 
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volume is "behavioristic" in the sense of approaching the topic of 
learning in the tradition of Pavlov, Thorndike, Hull, and other ob- 
jectivists. Bur it deviates from or transcends this tradition in that 
rious problems which the behaviorists 


it re-admits to consideration v: 
had renounced: hence its qualification as zeobehavioristic. 

Yet the position here adopted is not onc of mere conciliatory eclecti- 
cism. If, for example, we sometimes speak of “consciousness” (a tabued 
word for the Behaviorists), this is not just a friendly gesture to the 
past or concession to common sense; it represents instead the growing 
conviction that the objective study of behavior has now reached the 
point where some such concept is essential if systematic progress in 
theory construction is to go forward. Indeed, it is perhaps not too 
much to say that we have now reached a point at which, if con- 
sciousness were not itself experienced, we would have to invent 
Some such equivalent construct to take its place. Behaviorism was 
thus no idle “detour” or meaningless aberration. It was instead a new 
foundation which had to be laid before a sounder total psychology 


could be constructed (see also Mowrer, 1960). 


IIl. Learning Theory: Limitations and 
Possibilities 
Perhaps we can say that only now, half a century later, after the 
of Behaviorism, are we in a position to embark upon 
some such approach to the problems of psv thology d that envisioned 
by Angell and the other carly Functionalists. But if the body of 
objective observation and theory accumulated during the era of Be- 
haviorism has provided a new and sounder basis for attacking cer- 
tain problems, it also warrants à word of caution and circumspection, 
Probably the greatest single accomplishment. of this movement has 
: Cheah MNS : à 
; 3 rning theory, as we know 
been the gradual develo nog à SUN dU 
today, And although it i od iE El 
aspirations have been expressed for it, as a basis upon which many 


: D A tter solutions, There is 
other types of problems may find new and bet ; There is, 
: which learning theory is severely limited. At 


of the basic, biologically determined prin- 


rise and decline 


pment of les 1 
s still in its formative stage, many hopeful 


how ever, a sense in 
best, it gives us a picture : : : 
ciples, or Jaws, of learning. It has little or nothing to say about 
What living organisms, and human engs i pwned dug to learn 
Or What they in fact do learn: yet this is a question with which parens, 


teachers, ministers, jurists, 


and many others are deeply concerned. 
arni - he d 'rsons, once they have deci 
Learning theory may help such pe À cided 
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upon their objectives, to attain them; but it never, or at least only 
rarely, dictates what these objectives should be (cf. Skinner, 19533. 
and Jourard, 1958). 

In a paper entitled, “Ethics and Theories of Motivation and Learn- 
ing." Professor B. J. Diggs (1957) has put the matter well in these 


words: 


In regarding human action from the standpoint of learning theory, we 
have been looking at moral problems primarily from the standpoint of the 
individual. The means most essential to the promotion of individual goals, 
however, as well as a primary determinant of those goals, is the set of inter- 
personal structures collectively designated as society or culture. The family 
or some substitute therefor is essential to the maturation of the infant and 
the elementary education of the child in the w ays of the group; it supplies 
the "learning" originally lacking. In various ways, other { groups] continue 
the education begun by the family circle, until finally the matured indi- 
vidual is given the Opportunity of selecting, or is assigned to, some position 
from a set [of "positions" or “roles” | which his culture makes available. 
The ultimate acquisition of such soc status is even from the standpoint 
of the individual the most essential condition in his having the means at 
hand for the gaining of his goais, however narrowly these goals are. con- 
ceived. Thus society or culture provides the individual with sustenance, 
protection, and education, all of which culminate in social status; and in 
the process it gives the individual (beyond his primary drives) his char- 
acteristic wants, a knowledge of how these wants can be satisfied, and actual 
means-at-hand. At the same time and, if vou like, in return. therefor, it 
imposes rules to be obeyed. Just as individual learning must follow rules, 
so society must embody rules—if either of them is to be goal-attaining. All 
of this is common-place. 

The so-called distinctively human values are no doubt products of this 
social learning; or if we regard societies themselves as the products, these 
values are the products and by-products. Such values thus arise from man's 
ability to learn, but this ability itself is strongly conditioned and is brought 
to fruition by the kind of society into which the individual is born. Among 
these values are classes of valued objects, sets of valued courses of ac ion, 
and sets of norms—all of these expressing the “habits” or the "learning" of 
the group. Justice, conceived as a general virtue, consists simply in obedience 
to the rules. This might lead one to interpret statements of obligation “You 
ought to do X" (at least a large class of them) as commands expressing 
the rules of the group. The most important of such statements command 
categorically. They don't prescribe action contingently, depending on what 
the individual wants; they don't even say “If vou wish to remain in our 
Society, you ought. . . ." Since the actions presc ribed by the rules are es- 
sential (or are assumed to be essential) to the society, the rules simply 
command. Behind the commands, and expressing them, are the institutions 
M Du ad io uM, force of all individual pursuits of goals, 

) Sr ary. society says to the individual, “We have 
given you your ability to see; do not lose sight of us." If it be asked “Why 
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as Dewey puts it, 
aping such cate- 


“ this in one sens 
The only way of 


are men subject to these commands. 
Is equivalent to asking “Why liv 
gorical commands is not to be born (pp. 106-107). 


It has often been pointed out that man has, and is a product of, 
two great heritages: the one biological, the other social, His biological 
heritage is carried by and. transmitted through the chromosomes he 
receives from his progenitors; whereas his social heritage, as Professor 
Diges indicates, is the culture of his group, and this is acquired, extra- 
organically, through learning. But the capacity to learn is biologically 
given; and it is with this sct of. innate processes and. principles that 
learning theory is most. directly. and importantly concerned. In this 
5 > ticular consequence what an organism 


frame of reference, it is of no p: 
learns: the basic question is how ne 

“Personality,” or what a particular human (or infra-human) or- 
“result. of his particular social and other experiences, 
is thus beyond the scope of learning theory, 
: arch on learning is still at 


p 


ganism, as a 
eventually becomes, 
strictly: speaking. Much laboratory re 
the stage of using "naive" subjects: rats and other small mammals Or 
birds ú hich have been permitted to learn as little as possible before 
coming into the experimental situation, Only in this Way, it is 
‘a biologically given laws of learn- 


reasoned, can we get at the basic. l di d : 
ing. Hf we study the learning, or problem-solving, abilities of a highly 
“acculturared” ‘organism (such as man or, to a lesser extent, a trained 
or "pet? eins our results may easily be confounded: we cannot 
c to do "on his own.” as opposed to what 


he can do through culture. Culture is an accumulation of “answers” 
or problem solutions; and if we wish to study the problem-solving 
we do not want a subject who has learned a lot 
through instruction. from or 


tell What the organism is abl 


process in the raw, 
of problem. solutions vicariously, 1.6. 
the example of others. 

Great hopes have often bee 
theory as the master science 


n expressed in recent years for learning 
af psychology. A knowledge of the 
by we learn is undoubtedly necessary 


Congenital mechanisms where i 
anding of human personality; vet one won- 


ders if, on the basis of such knowledge alone, onc could ever 
deduce or predict either the development of personality ina particu- 
lar individual or the development of culture, historically speaking. 
With Professor Diggs, we may 
distinctive institutions, practices, 
nomena, i.c., compatible with but n 
dent conditions (cf. Kroeber, 19: 


for a complete underst 


rightly wonder if many of man’s 
and values are not emergent phe- 
ot rigorously determined by antece- 
White. 1949; and Childe, 19s is 
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1955). It is generally agreed by anthropologists that human beings are 
today probably not very different, organically, from what they were 
when they first started using language and accumulating and transmit- 
ting “culture,” f Y 


» Query: If all we 


tv or a hundred thousand ve 


now know about the learning process, or can reasonably hope to know 
in the next several decades, had at that point been put into a “giant 
computer" or "logical-consequences machine," could the machine 
have deduced or predicted anything very close to What has been the 
actual course of human development during this period of time? 
Answer 


Perhaps—but very probably: not! 

Learning itsclf makes for a sort of indeterminancy in the behavior 
of living organisms, i.c. it makes them, fortunately, changeable. We 
assume, of course, that there is a certain lawfulness about this change- 
ability, this capacity for being modified by experience. But no 
matter how completely we know bow learning takes place, it does 
not enable us at all accurately to predict &cbat will be learned—much 
less azbat ought to be learned. Miller & Dollard (1941) have put the 
matter picturesquely by saving that in order to understand or predict 
what a rat will learn to do in a maze, onc has to “know both the rat 
and the mi 


ze.” ! In other words, on the basis of learning principles 
alone, one cannot at all accurately foretell what an organism will or 
will not do; one must also know what both his internal environ- 
ment (tissue needs) and his external environment ( physical and 
social) are going to be. A psychology: of the total person must, there- 
fore, be anchored no less in sociology, history, culture theory, and 
even geography than it is in biology. While learning capacity has 
made both society and culture possible, learning, as this term is usually: 
understood, refers to biologically determined processes and, in and of 
itself, is not at all adequate to "explain" personality, culture, society, 
or any similarly complex deriv: ` 


\ ive of human experience. 

At the same time, these disclaimers are not meant to be taken too 
ategorically. For example, the distinction. between learning as a 
Process, as > "onte i üintain. 

I ? opposed to content, is not always easy to maintain.” 


iS 
Sears (195 > i z i 
kir om P has put the matter this way: "Changes in behavior are of two 
A a d . d * H a r 
theory to be dynamic, both must be systematized, separately but 


congruently. 5i i i 
w eid Nas aes action, or performance, and the other is learning 
M. ss Pu ini : 
he combining of these two approaches to behavior has not 


yet been fully acc otis! 4 € » 
ccomplished 8). S co! 3 
l 7 € also Scars, Mac Vo Les: (19357) 


eau otis, s and historically sophisticated discussion of the relationship 
y of process and content in psychology, see McClelland (1955) 

dicia i under the sway of litche 1 E i 955). 
tuch concerned with content but th 


He note 

s that, 
ner and the Introspectionists, psychology 
at, later, it became almost exclusively pre- 
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Harl 
low (1949) and others have recer tly reported convincing work 
à ^ D 


eA nena id : 5 
the acquisition of w hat they call learning sets,” or "learning to 


learn," i TOS ; 
rn," in monkeys and other mammals. And, as indicated elsewhere 


(Mow rer, 1960), the learning of language by a human infant open 
up for him further learning capacities or at least opportunities. Mur 
he learning theorist, as such, is not properly 
i.c, with questions of what any given person 
. holding that these are 


over, in insisting that t 
concerned with values, 
ought to learn or teach, we are not, 


not entirely legitimate concerns for other 
at other times. Behavior theory is a 


of cours 


psychologists or even for 


the erstwhile learning theorist, 
somewhat more inclusive term than is learning theory; and personality 


more inclusive still. The point is that the psy 
as a learning theorist must be allowed 


theory is, of course, 
chologist when functioning 
to define for himself an appropri 


able task." 


ately modest and scientifically manage- 
» . o 


Now, he suggests, there is a revival of interest in content 
thar we can have anything like a psychology 
devolves upon the selection of 


` failed to work 


occupied with process. 
and that it is only by studying both 
of the total person. The issue, McClelland holds, 

the proper contentual categories. litchener's “content categorie 
“not related to ex 
r on the other 


velopment? (p. 301). 


TM perimental operations on the one hand or to 
. For these or other reasons, they simply did 
“If ps "chologists are to re-enter the 


ies of genuine 


other types of behavio 


not lead to theoretical de 
field of mental content and start classifying it according to categor 


theoretical fruitfulness, | fear they will have to return to disciplines they have 
religion, art, histo economics, politics? 
are on the brink of an important new development. in 
ethodological improvements, We are about to take up 
ntal content that formerly were considered to be 
psychology” (p. 302). It is believed that the type of learning 
shall follow in this volume, while not itself dire dy 
is nevertheless logically hospitable to the con- 
clusive emphasis upon process. 

between the sciences of genetics and 
arily thought of as concerned 


(p. 302). “IE my 


long neglected . - - 
qa. wc 


analysis is co 
psychology. Because of m 
again some of the problems in me 
an essential part of 
theory whose development we 
he content. issue, 
yposed to an ex 
to the relationship 
Genetics is ordin 
posed to their environments. Yet it is well 


nent, especially during the embrvological period 
: i ud d, 


vs medium or milieu, quite as much as upon its 
e development irrespective of énviten: 


concerned with t 
cept of content, as 0] 

"Passing reference 
may be helpful here. 
of organisms, 35 op 
velopt 
sn 


aot determin 
the individual will react, developmentally, given 
ally, give 


Yet genetics has justified itself as an independent 

with learning theory, in contrast to the more 
will be apparent. A thoughtful AE 
nt in this connection: “This is another em 


embryology 
with the heredity 
known that individual de 
is dependent upon the organi 
The latter dot 
determine how 
itions. 


chromosomes. 
ment; rather they 
more or less favorable cond 
and distinet science. The analogy 
of psychology, 
following comme 


of regarding t 
model), one should regard the organism roger! 
5 Serer 


substantive" areas 
student has made the AS f 
he organism s behavior ‘as a function. of 

o 


of saying that instead 
Hull-Spencee 


Variables’ (the 
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As briefly indicated in the Preface, it is believed by some. including 
the writer, that the continued. pursuit of knowledge concerning the 
psychology of learning is scientifically and socially worth while. Al- 
ready it has been of value, practically. in various minor w avs, and 
eventually it may vield more impressive dividends, However, the 
student as well as the specialist in this field does well to keep in mind 
the strictures set forth, a decade ago, in a paper entitled "Scientific 
Models and Human Morals” (Allport, 1947). Here the author argues 
that we have been 

. 80 stupidified by admiration of phy sical science that we believe psy- 
chology in order to succeed need only imitate the models, postulates, 
methods and language of physical science (p. 182). i 

The machine model in psychology had its origin not in clinical or social 
experience, but rather in adulation of the technological success of the phy si- 
cal sciences. Since psychologists, like everyone else, are enmeshed in the 
prevailing ethos, they too, unless especially on guard, are likely to allow 
their subservience to technology to outrun their moral sense (p. 183). 

The foregoing was written. before the principles of mechanical 
automation ‘(Sev bernetics” ) were widely known and is more strictly 
applicable to those less “intelligent” types of machines which were 
developed during the 19th Century and which were indeed "mechani- 
cal" and lacking in the capacity for self-regulation and self-direction 
le ways. In. later 


which living organisms manifest in such remark 
chapters we shall sce that contemporary learning theory posits 
principles more like those involved in modern servomechanisms than 
those of the mere "pow cr-transmitting" machines of the last century 
(sce also Mowrer, 1960, especially Chapter Se 

However, Allport warns not only against mechanical models in 
general, but also against the "phylogenetic" (animal) and the "in- 
fant" models. In concluding his paper, he says: 


The designs we have been using in our studies of motivation, of symbol, 
and hence of the foundations of moral behavior, are not—to borrow 
Morris's crisp term—sufficiently iconic with our subject-matter. Addiction 
to machines, rats, or infants leads us to overplay those features of human 
behavior that are peripheral, signal-oriented, or genetic. Correspondingly 
it causes us to underplay those features that are central, future-oriented, 
and symbolic (p. 190). 


with its surrounding environment, physical and social, as a single system under- 
going various states through time (thermodynamic model). Comparative psychol- 
ogists have urged a similar position for some time.” However, for present purposes 
we shall adhere to the more conventional mode of analysis, which assumes the 
distinctness of organism and environment, the principles and the results (content) 
of learning, the how and the whar. 
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Here. in the present volume, we are by no means assuming that 
man is "just a machine,” or "just an animal," or "just a big baby." A 
normal, adult human being is manifestly “something more,” a truly 


transcendant creature, so that, admittedly, none of these models is 


at all. fully "iconic." 
adult man indubitably shares with the infant, the animal, and, quite 
of machines. And since, as indicated 


However, there are certain processes w hich 


possibly, with certain types 
in the Preface, we are here avowedly using the Approach-from- 
should operate at these lower levels. We 


Below, it is proper thar we 
of avoiding the reductionist’s fallacy and 


must simply make a point 
are trving, not to pull human nature down 


of remembering that we 
l| of conceptualization, but rather 


to a lower and inappropriate leve 


to build our conceptual scheme up to the point that it will con- 


nect, naturally and meaningfully, with human personality in all its 


complexity and wonder. 


IV. But Why Learning “Theory”? 


feature of all science. But the area of 


Theory, to be sure, is a : 
ith the learning process—or processes— 


psychology which deals w p 
seems to be especially characterized by theory and theorizing. Why 
should this be so? x 

In the psychology of sensation, theories are, ti 
lacking; but it is notable that the expression, “sensation theory, has 
no great currency; Whereas one is today more likely to speak of 
1 ^ than simply of “learning. Perhaps this au 
are really more interested here in 


of course, not entirely 


“learning theory” 
oblique w av of saving that we 
what learning theorists are. themse l | 
the “thing itself." At first brush, this strikes one as a rather singular 
situation. But a moment's reflection will make it more intelligible. 
Everyone has sensations; on the basis of immediate ex- 


Ives thinking and saving than in 


everyone, 
perience, knows what tastes, odors, colors, and sounds are like. 
But it seems that no one. ever has a “sensation” of learning." There- 
fore this phenomenon has the status of a construct and is knowable only 


inferentially. Te is capable of being scientifically studied and even 


Iwo or three decades ago a mov ement, known as Gestalttheorie, tried to de- 
of learning based on 
"ah-ha^ experience. 
Gestalttheorie has not been fruitful in the learning 


xplaining insight (or so-called “discontinuity” ) 


the premise that learning is directly 
Although sound and scientifically 


velop a psychology 
knowable, as “insight” or the 
productive in some of its tenets. 
area, We are today much nearer to € 


in terms of learning principles than we are A i : 
Pa alt principles. (See Mowrer, 1960, especially Chapter 6). 


to being able to explain learning Cecon- 


tnuity™) in terms of Gest 
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casually observed in terms of its conditions and consequences; but 
the operation itself is completely "silent." Everyone knows a good 
deal about what he has to do to learn this or that, and we all have 
rough ways of testing ourselves for learning. But never, apparently, 
do we have any awareness of the “wheels going ‘round’ " as ae learn. 

In the evolution of living organisms, there was presumably no 
premium on their being directly conscious of the learning process. 
Just as there is ordinarily no gain in brain tissuc being “sensitive 
to pressures or lesions (since the skull usually. protects ‘the brain so 
adequately), so likewise was there no reason for living organisms to 
have a sensation. whenever the learning process takes place, But this 
fact constitutes a problem for psv chology and was certainly one of 
the shoals on which Structuralism foundered. If le: arning is accompanied 
by no distinctive sensation or state of consciousness, then it is not 
open to investigation by means of uid sata Accordingly, Struc- 
trualism gave us no psy chology of learning, no * "learning theory," and 
asa total psychology was not Viable. 

As we have seen, Functionalism accented behavior in a w ay which 
Structuralism had not. But, at the turn of the century, hypotheses 
and methods for the study of behavior were still primitive and v ague; 
so it is understandable that the Functionalists and early Beliasivsrists 
should have seized upon the concept of the reflex. Various methods 
for its study had been developed by the physiologists. But, by tradi- 
tional definition, one of the salient and identifving features of a re- 
flex is its invariability, its mechanical regularity and dependability. 
Yet the characteristic and distinguishing thing about most behavior, 
and especially those forms of behavior in which psychologists are 
especially interested, is its flexibility, fluidity, “freedom,” non-fixity. 

In an attempt to remedy this situation, psychologists were obliged 
to turn their attention, more seriously than ever before, to the phe- 
nomenon of learning. Behavior is clearly and manifestly subject. to 
modification, change. How, precisely, does this come about? That 
was the question. 


Two answers were soon forthcoming. The work of L P. Pavlov 
(1927)—a Russian physiologist who, in studying certain “digestive 
reflex S" became impressed. with their modifiability —showed onc 


way (for Pavlov the only w ay) in which behavior can be modified, 
“By the same token, learning lent itself 


which Behaviorism, or S 
obsery 


particularly well to the type of inquiry 

fostered. “This approach presupposed no 

mental state but rather a change in the relation- 
s between conditions and consequences, 


R psychology, 
ation or knowledge of any 


ship between stimulus and response 
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i.c. by so-called cozditioning. If a formerly neutral stimulus is paired 
a few times with a stimulus that dependably (“reflexly") produces a 
given response, soon the erstwhile neutral (ineffective) stimulus will 
be capable, alone, of eliciting this response. The response, in other 
words, will have been “associatively shifted," to borrow an apt cx- 
pression from Thorndike, to a zezz stimulus (see Fig. 171). This and 
this alone, Pavlov averred, was the basis of behavior flexibility and 
adaptability. Bur, it will be noted, responses which had been thus 
conditioned were still. reflexes, “conditioned reflexes.” 


S. 


3f 


S,———R 


1 
“as it was conceived by Pavlov. The three 
ft to right. Initially, some response R can 
equate,” or “unconditioned,” stimulus 
and closely associated with (preceded 


Fig. 1-1. Learning, or “conditioning, 
little diagrams are to be read from le 
be elicited only by its “physiologically ad 
5. But it may happen that Si is regularly i es | 
by) some other stimulus Se Asar ilt of this association, conjunction, or "tem- 
, “associatively shifted" from S; to S. 


poral contiguity,” response R gets 
The biological utility, ie. the advantage to living organisms in 


their strugele for existence, 
ic ate forthcoming events and to make more in- 


reactions thereto (cf. Chapter 10, Section I). 
Pavlov and his numerous co-workers and students, both in Russia 


and in many other countries. obviously “had something.” but did they 
clearly thev did not." One of the most 


at so-called conditioned reflexes, 


of the conditioning process is clear: it 


enables them to azticip 
telligent, more adaptive 


“have anything”? Pretty 
palpable difficulties was the fact tha d 
instead af being exact replicas of their unconditioned prototypes, are 


often perceptibly: and, sometimes, radically different. Specific in- 
stances of this sort will be cited in connection with the discussion of 
avoidance learning in animals in Chapter 3; and the phenomenon is 
likeli ta betre "more conspicuous at the human level. For example, 
al years ago Wickens (1938) showed that if human subjects are 
E threat of electric shock to lift a finger, 
hich the finger has been lving palm 
also lift the finger, by flexion, 
has been placed on the 


seve 
trained ("conditioned") under 
by extension, from a grill on which 
with no further training, 


down, they will, : 
at the appropriate signal, when the finger 
"For a detailed account of the status of Pavlovian reflexology. both as science 


and as ideology, in Russia today, see Razran (1957). 
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grill palm up! Here, in the latter situation, the response of lifting the 
finger off the grill involves an enurely different set of muscles and 
motor nerves from those involved in the original training, s 
can hardly characterize such a response as a 
Indeed, common 


0 one 
"conditioned. reflex." 
sense would suggest, and laboratory observation 
would probably confirm, that if, after such training, the 
foot were placed on the grill. it might 


ght well be lifted when the 
danger signal occurred. Or, a subject so trained. might even shout 


a warning to another person who happened to be 
the signal came on. Clearly the 


subject's 


near the grill when 
\ paradigm of learning offered by Pavlov, 
while intuitively recognized by everyone as having some validity. 
has serious limitations and does not provide a master formula for the 
interpretation and prediction of all behavior. 
The other carly attempt to account for behavior modifiability 
was, of course, that made by E. L. Thorndike (1898; sce also Jennings. 
1906), with the concept of “habit.” g 
where the conditioned reflex. idea was weak, in that. it accounted, 
at least within limits, for response variability. 


"phis concept was strong precisely 


The basic notion here 
is that living organisms have drives, such as hunger, thirst 
and that the response first made to a given drive in 
may not achieve the desired ends. Therefore, living 
have provision for re 


. and cold, 
à given situation 
organisms must 
sponse substitution, i.c., some means whereby 


Fig. 1-2. Thorndikian habit. formul 
stimulus S, characteristically 
R, has served to re i 
longer “works.” 
(middle diag 


ation or trial-and-error. lea 


ning. Some drive 
produces some response Rj, because in the past 
duce and gratify the drive. But conditions change and Ry no 
The organism is thrown into variable, trial-and-error behavior 
t ram) and "tries," in succession, responses Ry, Ra... Ra the last of 
which produces the desired effect. On successive repetitions or trials, R, gradually 
rises in the “response hierarchy" (Miller & Dollard, 1941) until, as shown at the 
righr. whenever Sa occurs. R- promptly and specifically: follows. “This type of 


behavior r cari 
: modification has often been called response substitution or selective 
learning. 
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à reaction made originally, but inappropriately, to a given source 
of stimulation can be replaced by a more effective one. This, for 
Thorndike, was the gist of habit formation (Fig. 1 

We have already seen how Pavlov's theory of conditioning ar- 
able to learn to react 


ticulated with evolutionary thought: in being 


to signals, as well as to the things signalized, living organisms have a 
better chance in the struggle for existence. But, as Campbell (1956) 
has pointed out, ‘Thorndike’s theory also shows the imprint of 
in fact, a theory of the evolution 
In so-called organic evolution, 
sing more or less 


Darwinian influence. lt was 
Of acts, as opposed to individuals 
the "trials" and the "errors" are individuals pos: 
advantageous structural. characteristics, and the competition is be- 


tween these individuals; whereas, in learning, according to Thorn- 
dike's view, the competition is between different response tendencies 
within the sime individual, and learning is the process that ensures 
"survival" of the effective and. elimination of the ineffective ones. 


As Campbell observes: 


andom variations is taken as a general paradigm 
o environment. Darwinian theory of natural 
selection applies the model to the fit between the gin PER i 
Of organisms and the opportunities provided by her du sed taland, 
error doctrines apply the model to learned fit between organismic response 


and environment (pap h 


_ Selective survival among r 
for instances of organismic fit t 


Naturally enough, Thorndike made much use of this latter means 
of behavior modification. Although he did not deny the reality isi 
conditioning, he attached, as we shall later see, little significance to 
it. But “habit” had its troubles, too. Whereas conditioned-refles 
stimuli to established (unconditioned, 


theory: spoke of attaching es : DURS 
or unconditional) responses, habit theory spoke of xn ling, or con- 
Necting, established stimuli (drives. needs) to new responses. In other 
» response remains constant (in theory) and 
whereas, in habit formation, the stimulus 
remains constant and the response changes. As between these two 

at nothing was left to be desired. But 


Mechanisms, it might seem th i 
E akness of being modeled on the 


both suffered from the common wc di 
flex.1! As already noted, a reflex modified by con- 


Words, in conditioning the 
stimulus potency. changes; 


concept of the re 
ation" and "natural selection" and expres- 
. Miller & Dollard, 1941) 


Cf. also Darwin's terms "natural v 


“Mons employed by contemporary learning ts (c 
‘ ! "response selection. 


cterizing his theory of learning as 
his theory as being connectionistic, 


theorists (e.g 


such as “response variation" and 


Pavlov was quite open and direct in cha 
reflevological; whereas. Thorndike spoke of 
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ditioning is still, at least by definition, a reflex; and a “reflex” moditied 
by the kind of learning called habit formation is, also, still essentially 
reflexive. The latter point can be simply illustrated. 

Critics of habit theory have often pointed out that it makes all be- 
havior, whether learned or unlearned, "mechanical," "blind," and, 
of necessity, more or less blundering; whereas the truth seems to 
be that, at least on occasion, living organisms can be exceedingly 
clever—foresightful, intelligent, resourceful. Pur a little differently, 
the objection is this. Thorndike, on the basis of his theory, popularized 
the notion that we “learn only by doing." This follows, directly 
enough, from his theory, which savs, in effect, that before an existing 
"habit" (or “reflex”) can be modified, i.e, before its chances of rc- 
currence can be cither increased or decreased, it must occur and be 
cither rewarded or punished. There is, in other words, no possibility for 
the organism to be “smart,” and do the “right thing" in the first place. If 
a given response is wrong and will be punished if performed in a given 
situation, there must be at least one error (perhaps several), savs 
the theory, before the organism can "mend its w ay 


" And if a ré- 
sponse is the correct onc for a given situation, it must likewise 


occur onc or more times before the organism can have increased 
confidence in it. Hence the familiar characteris 
tion as trial-and-error learning. 


ation of habit forma- 


Now it is only too evident, alas, that those creatures with even 
the best brains do far more foolish things than we (or they) might 
wish; but it is also true—habit theory notwithstandine—that some 
of the time living organisms are able to by-pass even that ove 
trial, or “error,” and behave quite adaptively: (i.c. change, learn) 
without "doing." Here, then, is habit theory's Achilles’ heel, into 

arrows have, in the past few decades, been deeply 


Which a number of 
driven.!? 


Clearly the distinction is more terminological than real. Woodworth (0918), writ- 
ing ata ume when he was much under the sway of V horndike's views, said: “A 
Han carries around with him a vast assortment of possibilities of action. The best 
conception of a ‘possibility of action’ is undoubtedly that of a neural mechanism 
s connected with other neural mechanisms and w ith the sense organs and muscles 
orae 5 an Pond (p. 106). One could hardly formulate a better 
vn iei dise E se ever, granting that lhorndike's theory of learning 
Mob soe, sicci t = was thar of Pavlov, the fact remains that the one 
“hfe melas oe a € ^ A eA ge stressed stimulus substitution. 

sates Paan i eet xa xy 1 of ne vogue w hich Gestalt psy chology, 

Pty enjoyed in this country during the 1930's and 


arose fre | manifi 
"ipa m the manifest adequacy of the notion that w 
Pestalttbeorie was itself critically weak in 


194078 
€ learn only by doing. 
certain important respects; and 
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V. Methods and Objective 


SMS the preceding section indicates, neither of the two conceptions 
of learning which were most influential during the first "uw 
this century provided a wholly satisfactory explanation of all feet 
Like the corpuscular and the wave theories. of light (in ilios. 
these. two conceptions of learning were apparently: both TERY 
to account for different types of experimental and commonplace 
data; so it was natural that many investigators should have concluded 

5. a View 


that learning involved two distinctively ditferent. processes 
which was eventually dubbed trvo-factor learning theory (Mowrer, 
1947). In the next two chapters we shall see how, between 1935 
and 1950, this view was elaborated and systematized. But then, in Chap- 
ters 4 to 6, we shall see that, while an advance over the classical views 
of Pavlov and Thorndike, considered separately, two-factor theory was 
likewise unsatisfactory: in certain. important: ways, including its in- 
ability. to account for a form of so-called secondary reinforcement 
which was being increasingly studied from about 1945 on. 

This work on secondary reinforcement. led to a major revision 
of two-factor theory, which was enunciated in 1956 (Mowrer). Here 
an attempt was made to show that, in the final analysis, all learning 
can be reduced to conditioning, if that. process is properly defined, 
However, according to this view, learning is still “two-factored” 
in the sense that it occurs under two different conditions of rein- 
forcement; drive reduction (reward) and drive induction (punish- 
ment). Thus, instead of invoking Pavlovian theory to explain certain 
instances of learning. (so-called conditioning) and Fhorndikian theory 
to explain other instances (so-called. trial-and-error learning, or habit 
the attempt is made to integrate these two conceptions 


formation), 
into a more pow erful and truly united conceptual scheme. 


This new conceptualization carries us far, as will be apparent in 
Chapters 7 through 12. But, in the sequel to this volume (Mowrer, 
which deals with the wearious learning w hich language makes 


1960), 
possible, it Is necessary to come to grips Ww ith the problem of semantics. 


Here it is found that the meanings of words have both a connotative 
(evaluative, good-bad) and a denotative (or "cognitive") dimension; 
and while the revised version of two-factor learning. theory can 
as SR psy chology has become more sophisticated and more adequate, there has 
been a steady falling-off of interest in this alternative approach (see Chapter 9; and 
Mow rer, 1960. Chapters 2 and 6). 
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readily account for the evaluative aspect of meaning (through the 
association of words with the experience of reward or punishment). 
it cannot explain, without further development, the purely denotative 
aspect of meaning—or certain familiar aspects of memory. In Chapters 
s and 6 of this book and in Chapters s—> of Learning theory and tbe sym- 
bolie processes (Mowrer, 1060). an attem 
this complication; and, as a result, 


pt is made to work through 
it is discovered that psychology is 
today again. concerned, although under a new 
those banished “faculties,” or “mental 
cognition (knowing) 


terminology, with 
functions" once. known as 
. conation (willing). and affection (feeling y. 
However now, instead of 1 


avino either a psychology without. learn- 
ing (as we may characterize Structurali m) o 


r learning without psv 
chology (radical B 


chaviorism), we have a conceptual scheme in which 
learning and the more distinctive “mental” functions are integrally and 
inseparably: related. 

As is thus apparent, the approach to be followed in this volume is 
heavily historical; but it is one which cztches in its sw cep most of the 
great issues and problems which have ox 
and investigy 


upied the creative thinkers 
ators in the field. It is hoped that the reader will find in the 
ensuing account of these developments. some o 
fascination which the field has held for 
immersed in it. 


f the challenge and 
those who have been personally: 


The Law of Effect, Conditioning, 
and the Problem of Punishment 


The preceding chapter has indicated something of the influence 
Which F. L. Thorndike and L P. Pavlov were to exercise on the develop- 
ment of learning theory during the first half of this century. In 
the present chapter we shall be concerned, more specifically, with 
the problem of punishment and with the contribution which the 
igarors and the research they have stimu- 
T understanding of the problem and, in 


more comprehensive, more self- 


Views of cach of these invest 
lated. have made to the better 
this, way. to: che development of a Í : on 
1 theory of. learning in general. This may 
av of approaching the psychology: 
tive" aspect; but this is 
Moreover, such a course 


Consistent, and more unified 
seem an awkward and perverse Way 
Of learning, to begin with its more 


how the matter. has moved historically. à 
h certain logical advantages to a con- 


“ne 


leads quite naturally and wit 
sideration, in later chapters, of 
(see also Mow rer, 1960). 


the psychology of learning in its totality 


I. Thorndike's Early Position Re-evaluated 
and Revised 
: x dermal F " 
^ possible source of misunderstanding should be dispelled at the 
Outset Here we are to take. as basic, à conception of punishment 
Which ‘Thorndike himself later repudiated—or 
toned. Such a scrategy; without explanation, might suggest that a 


at least. seriously ques- 
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“straw man? was being set up only to be knocked over. Such 
is not at all the case. 

"Vhorndike's original position was to the effect that learning is a re- 
versible process, reward strengthening and punishment w cakening it. 
However, as a result of 


a long series of experiments on verbal learn- 
ing in human subjects carried out later in his professional life, Thorn- 
dike (1931. 1932b) came to the conclusion that his original. position 
had been in error and that, while reward does indeed facilitate learn- 
ing, punishment does not weaken it. The details of this remark- 


able interlude in T horndike's thought have been carefully: review ed 
by Postman (1947) and need not be repeated here, but it is relevant 
to note some of Postman’s own comments on this score. 


Thorndike’s revised view of punishment startled the psychological pub- 
lic. Not only did it contradict the belief in the practical value of punish- 
ment which had become almost an axiom of our social life; it was also 
clearly at variance with a considerable body of other experimental data 
Cp. 502 ). 

When we turn to punishments other than an announcement of wrong or 
slight monetary losses, the uncertainty of “Thorndike’s generalization. be- 
comes even more apparent. . . . As against the blanket assertion that pun- 
ishment is not instrumental in the elimination of the w rong responses it is 
possible to cite a long list of papers covering more than half a century. of 
experimental work (cf. also Thorndike, 19322]. which report. that punish- 
ment is an effective condition of learning (pp. 505-506). 


When Thorndike moved from his earlier, “classical” theory to the 
new position (which was to be espoused and elaborated by Hull and 
his followers; see Chapters 3 and 8), he was clearly on less firm 
ground than he had initially been; so we shall here deliberately 
take as our point of departure his earlier formulations which, while 
admittedly incomplete and inadequate, at least give us a 
approximation to the truth. 

As Deese (1952) has remarked, “Thorndike, for many years, 
ominated the theory of punishment. Because his influence is still 
felt to a very considerable degree today, it is best to begin with his 
notions concerning the effects of punishment? (p. 112). In the end, 
t 


good first 


e 


dese notions broke down (or were abandoned), it seems, not be- 
cause they were wrong in any global sense, but because Thorn- 
ike refused to develop them in keeping with certain ideas which 
have been put forward, with much empirical support, by Pavlov 
and other students of conditioned-response learning. By taking I horn- 
dike's original a sm > 


: rather than his revised theory as a starting point and 
by Incorporating rather than ignoring the work of Pavlov, we find 
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ourselves on a road which promises to carrv us far toward the develop- 
ment of a more nearly "whole" conception of the topic under con- 


sideration, 
bY Ber P $ 

Perhaps, therefore, the quickest way to come to the heart of 
the issues with which we are here concerned will be to re-examine 
Vhorndike’s early statement of both "sides" of the Law of Etfect. 
\\ riting in 1913, Thorndike said: 

When a modifiable connection between a situation and a response is 
ed by a satisfving state of affairs, that 
When made and accompanied or fol- 
gth is decreased (p. 4). 


made and is accompanied or follow 
connection's strength is increased: 
lowed by an annoying state of affairs, its stren; 


And by wav of amplification, ‘Thorndike added: 


By the strength of a connection is meant roughly the probability that the 
Connection w ill be made when the situation recurs (p. 3). 

The strengthening. effect of satisfvingness (or the weakening effect of 
annoy ingness) upon a bond varies with the closeness between it and the 
bond (p. 4). 

In a grossly empirical, de 
Most certainly valid: other 
: occurrence has led to rew ard 
as led to punishment. Such a bifurcated 
ongruent with common sense; and it 


behavior adaptive, in the biological 


scriptive sense, the Law of Effect is al- 
things being equal, an act is more likely 
to recur if its prior „and it is less likely to 
recur if its prior occurrence h 
principle is, to say the least, C 
seems. well calculated to make 
scheme of things. 


However, at another level of interpretation, the Law of Effect has 


critical scrutiny. For at least two decades 
"negative" half of the law. in its 


Just as rew ard was said to "s amp 


in" bonds, associations, or connections, punishment was thought to 


* such connections. He bond = 
i reversible process. Just as re- 


Come under increasingly 
now, it has been clear that the 
as miscast. 


Move molecular aspects, M 


al m ^ ar i M xg 
Weaken, or "stamp out, Leanne, ot 


ing," was chus, as already noted, : 


a forward direction, punishment. caused 


Ward caused it to proceed in 
It to recede. If rewards, so to Says opened neural pathways, punish- 


Ment closed them. And the onc action was assumed to be just as im- 


Mediate and direct as the other. » m 

Whatever learning may be in its "forward," positive phase, it be- 
c “thar unlearning. Or 
stamping out stimulus-response 


came increasingly clear “punishment.” is not 
a simple matter of obliterating, 
"bonds" Bur a more declarative 
learning could not occur until at 


approach to the negative side of 
least. certain forms of that ty pe 
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of learning known as conditioning were taken into account. And 
here our story starts with the phenomenon of fear conditioning. 

It has long been known, of course, that fear can be attached to a 
formerly neutral, or “indiferent” stimulus by associating that stimulus 
with some painful experience. W. A. Bousfield, writing in the steri- 


can Psychologist for 1955, documents this statement in an unusual wav 
when he says: 


Professor J. H. Arjona of the University of Connecticut, the leading 
authority on the Spanish playwright Lope de Vega, recently sent me a copy 
of his admittedly free translation of a story in i opes play El Capellan de 
la Virgen (The C haplain of the Virgin). This play was probably written 
in 1615. Accompanying the translation was the pointed observation, “This 
antedates vour scientific experiments by about three centuries.” 

With Professor. Arjona's permission, 1 am submitting his translation. for 
the possible edification of the profession. . . . It would appear that. the 
highly prolific Lope was somewhat of an authority on classical condition- 
ing. 

“Saint Ildefonso used to scold me and punish me lots of times. He would 
sit me on the bare floor and make me eat with the cats of the monastery. 
These cats were such rascals that they took advant age of my penitence. 
They drove me mad stealing my choicest morsels. It did no good to chase 
them aw av. But I found a way of coping with the beasts in order to enjoy 
my meals when [ was being punished. | put them all in a sack, and on a 
pitch black night took them out under an arch. First | would cough and 
then immediately whale the daylights out of the cats. They whined and 
shrieked like an infernal pipe organ. | would pause for aw hile and repeat 
the operation—first a cough, and then a thrashing. 1 finally noticed. that 
even without beating them, the beasts moaned and velped like the very 
devil whenever I coughed, I then let them loose. Thereafter, whenever | 
had to eat off the floor, | would cast a look around. If an animal approached 
my food, all I had to do was to cough, and how that cat did scat!" 


Here was a situation in which the character who is telling the 
story was not able to punish the voracious cats directly, lest Saint 
Ildefonso sec him do it. Yer he manages to inbibit their behavior, in 
a simple vet Ingenious wi av. He conditioned. the cats so thar they 
would be afraid whenever he coughed, a sound which he could ma ke 
innocently enough in IHdefonso's presence; and later, when the cats 
were about to advance and devour his food, this stimulus alone was 
enough to keep them at a distance. licere one cannot legitimately 
speak of the “stamping out, by means of direct punishment of the 


cats, of the habitual bonds that caused the cats to eat. the ee $ 
food; but such behavior 


! Was none the less etfectivelv controlled. 
And this observation sugo 


ggests that even in the more direct and simpler 


instances of punishment, inhibition i Is achieved in another TAa. 
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First of all it is to be observed that the procedure described in Lope's 
play made coughing one of the stimuli which the cats, in effect, "pro- 
duced" when they started to advance upon the boy's food. Now suppose 
thar, instead of this artificially introduced stimulus, we make use of those 
stimuli which are zaturally Correlated with the behavior in question, To 
do this we let the behavior itself start to occur and then, on the spot. 
punish the cats. Now, by the principle of associative learning, or condi- 
toning, already described, we would expect fear to become attached to 
these stimuli, so thar the special procedure described by Lope would 
not be necessary: as soon as the cats started to perform the punished 


act they would, so to sav, automatically (ic, without anyone's 
and remain at a safe distance. This notion 


importantly different. from Thorn- 
; and it is now generally conceded 
by the direct 


intervention) become afraid 
of punishment. is obviously and 

dike’s conception of bond erasure 
achieves. its. inhibitory effect, not 
but by the intermediation of fear. 
An action, previously strengthened by rew ard, which is followed by 
c stimuli, both internal and external to 
the organism, which, by virtue of their contiguity with the punish- 
ment, take on the capacity to arouse fear; and when the organism 
subsequently starts to repeat such an action, the resulting fcar pro- 
duces a conflict. with the drive or motive underlying the original 
act. If the fear is sufficiently strong, the act will, in consequence, be 


ast in some fashion modified. 

of the way in which ordinary punis qe 
momentous Consequences, First of 
the views of ‘Thorndike and Pavlov in a way 
men alone was able to do. Pavlov, for reasons 
discussion continues, never really 
in the problem of punishment, and Thorndike, al- 
and concerned about the problem, refused 
of conditioning. But if we combine 
with the problem of punish- 
rges—a hypothesis which 


that punishment 
stumping out of S—R bonds. 


punishment, produces certain 


inhibited, or at le 
This revised. conception 


Ment operates has had two rather 


all it “integrates” 
Which neither of these 
^ hich will gradually emerge as our 
interested himself in 
though fully aware of 
to take seriously the principle 
the principle of (fear) conditioning 
ment, a new and powerful hypothesis emet : \ 
not only provides a superior explanation of punishment (in both its 
passive “and active aspects. see below) but which leads ultimately, 
38 we shall sec, to a revised and sounder 
tion itself. 

. Moreover, the deve 
tegration of learning theory 
Peychoanalysis in particular. 


conception of habit forma- 


ketched opened the way for the 
and clinical p chology, in general, and 
"Thorndike's original theory of punish- 


lopments just sl 
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ment, while sound as a first approximation, made no provision for the 
manifest phenomenon of conflict, and contlict is the very essence of 
the clinical problem. Although in this book we shall be only inciden- 
tally concerned with the latter (sce C hapter 11; see also Mow rer, 1960). 
yet it is reassuring to know that we are pursuing a conception of learn- 
ing which leads naturally to such matters, rather than automatically 
excluding them or denying their very existence. 


Il. Punishment and Fear Conditioning 
Punishment thus emerges, not as one of two aspects of a single (re- 


versible) learning process. but rather as a subprinciple, or "applica- 
tion," of another seemingly sepa 


ate type of learning. This other type 
of learning is fear conditioning and, in its most general form, can be 
conveniently depicted as shown in Fig. 2-1. If fear, with its powerful 
motivational properties, is conditioned to stimuli which are response- 
produced, occurrence of that response will tend, as already indicated, 
to be blocked: the occurrence or continuation of the response arouses 
fear and its discontinuation causes the fear to abate. If, on the other 
hand, fear is conditioned to stimuli nor specific: ally associated with be- 
havior, resolution of the fear will very likely be best achieved, not by 
inaction, but by actioi—action which will climinate the fe: ir- producing 
stimulus or situation cither by removing the affected organism therc- 
from or by changing the situation. In the one case, 


1 Lc, Where the 
stimuli are response-produced, we arc likely to speak 


"punishment? 


Contiguous stimulus <a 
or “signal = E = 
Secondary motivation ee AU at 
(fear E ee 
Primary motivation 


(e.g. shock) 


Fig. 2-1. Diagrammatic representation of fear conditioning. This emotion is orig- 
inally clicited by some primary drive, such as electric shock; but if some other, 
formerly neutral. stimulus immediately precedes the primary drive, the fear re- 
sponse, after a few pairings of signal and drive, will become attached to the 
signal and “move forw ard," as shown by the dotted line. When this occurs, the 
fear response is said to have become anticipatory The above di: 


mati Ge 

i €, but very similar actual records can be obtained by using the G 
SKIN reaction, as an indicator of fear (Mowrer, 
1954. Ch. 6). 


am ds sehe- 
SR tgalvanic 
1938; Woodworth & Schlosberg. 
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and, in the other case, of "avoidance learning.” although it is now 


in the sense of averting painful, noxious 


evident thar “avoidance.” 
stimulation, is achieved in both instances through the arousal of fear 
and appropriate (though ditferent) reactions thereto, 

The term, “punishment,” is. usually applied to the procedure em- 
ploved when an organism, prompted by some drive such as hunger, 
mode of behavior which has previously satisfied this drive 


exhibits ; 
and an etfort is then made (usually by another organism) to disrupt 


or eliminate this “habit by the introduction of a competing drive or 
drives, Suppose, for example, that a hungry laboratory rat has learned 
to run to a small iron rod (or "spatula^) with a bit of mash pressed 
on its fattened end whenever this rod is inserted up through the grill- 
like floor of the apparatus in w hich the rat is located. The rat will have 
aining in getting food in this way, over a period 


received considerable tr 3 , . 
that, at the time of the demonstration, its 


of two or three days, so ; - 
< insertion of the food-bearing rod will be prompt and 


Now suppose that we have decided to "break" 
unning to and taking the food from this rod 
"torm of an electric shock of, sav, two 
at's feet from the grill floor of 


response to the 


speeitic (“habitua 
the rat of the "habit" of r 
by means of punishment in the 


seconds duration—administered to the r j : 
alternating current will be used, with a 


ohms. The charge thus imposed upon 
o a human hand, but 


the apparatus. (A 12 5-volt 
000 


limiting resistance of 1 " 
the erill is so slight that it is hardly perceptib et 
it is mildly painful to the rat.) 


One can easily surmise what would happen to the rat's behavior as a 
tis sort of. punishment; but let us re- 


result of the introduction of tl : 
actual demonstration 


produce here notes made in connection with an 


Of the kind described. 
tst trial: On insertion of food rod, rat dashed, ae OIN e eg m 
and, taking it in his forepaws. started au k vi p “pp eu at 
dropped rod and "dancec round on the gri a2 MONI: Ris A à: d 
2nd trial: After an interval of 2 minutes, IRE SS im 2 oie ^ue 
at immediately ran to the rod, took it in v9 S, a peg Snog 
Vhen shock was applied. rat dropped rod, danced much as before, and 


squeale i d 3 F ‘ 

js s v cR ime rat ran toward rod and tried to grab the food off it, 
With teeh E lees raking rod in forepaws. Probably got some food, but also 
Bot shocked. Danced and squealed. 


A % ike precedir 

4th trial: Very much like precet — a 

Sth tri i Ms ld run x ard food rod, then suddenly withdraw, pause 
x di H o 


a moment 1 then repeat this performance. The "contlict" between 
CDU, anc e à d IH WAR s 
tunger and fear of being shocked, was thus very evident [see Chapter i1]. 


ac trial. 
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After four or five successive advances toward and retreats from the food 
bar (without actually touching it), the rat came brietls to rest at some 
distance, and the experiment was discontinued. 


While instructive. such a demonstration. leaves some unanswered 
questions. What would the rat, in this particular demonstration, have 
done on the next few trials? Would it soon have been so thoroughly 

"beaten" that, at least for the time being, it would abandon the food, 
preferring to be safe rather than 7 SOITY CEAT ould experience in tak- 
ing food and “riding out" a weak shock so condition a subject thar it 
would later be undeterred by a stronger shock which, if used initially, 
would produce marked inhibition: 


These and many other intriguing 
questions can be asked—and, without too much trouble; answered 
(see, for example, Chapter 11); but the observations recorded are suf- 
ficient for our present purpose, which is simply to illustrate, in some 
detail, the essential features of so-called punishment and to indicate 
how this phenomenon can be accounted for by the fear-mediation 
hypothesis, in contrast to I horndike's stamping-out notion, 


Ill. Fear Conditioning and 
Active Avoidance Learning 


Through historical accident and carele 
up of speaking of “punishment” and of * ‘avoidance learning’ "as if the 
former were, itself, not also a form of * ‘avoidance. As seen in the 
preceding section, this practice is misle: iding and blinds us to the com- 
monality of the principles involved. Hence it is here proposed that 
so-called punishment be termed passive avoidance 
to "avoid" by not doing something) and that the contrasting form of 


avoidance (learning to do something as d o nicans 


g of avoidance) be 
termed active avoidance le: arning. It is with the latter that we shall now 
be especially. concerned. 


Imagine an elongated, narrow box, about three feer in length, a foot 
and a half high, and five or six inches wide or “deep.” Imagine also 
that the box is divided into two , 
middle with a small hole or * 
compartments. easily 


ssness, the practice has grown 


learning (learning 


compartments, by a partition in the 


‘doorway in it, In order to make the two 


distinguishable, one—let us say the one at the 


1 
Because the shock 


hunger fairly strong, 
quite 


used in the demonstration 
there was nor 
active between trials, 
dicated by Jumping 
ously. A more intensi 


Was comparatively weak and the 
much general inhibition. Phe rat remained 


but there was a tendency to “leave the field," as in- 
behavior : attempted esc ape, 


which had not occurred previ- 
€ shock would have 


altered this general picture. 
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observer's right—is painted white and is illuminated by a small electric 
light, while the other is painted black and is unilluminated. Moreover, 
the oor of the black compartment is made of wood, while the floor of 
the white, or right-hand, compartiment consists of a metal grill, through 
Which an electric shock can be administered by the experimenter. 
With a laboratory. rat at hand, we are ready to start a simple demon- 
stration of gerive avoidance learning. We put the rat into the apparatus 
and let it thoroughly. explore for four or five minutes. The rat, which 


we can observe through a one-way mirror which acts as the front 


Side of the apparatus, will move about sniffing, looking, and feeling 
With its vibrassae, It will thus explore first onc of the compartments 
and then the other, passing. back and forth. between them until its 
“curiosity (see below) has been satisfied. Then it will settle down, 
Perhaps groom itself, and give the appearance of being just a bit bored 
With ir all, 

Vhe following are notes from an actual demonstration of this kind. 

Ist trials Rat soon came to rest, quite conveniently, in the w hite com- 
Partment, | Otherwise, after exploration, rat would have been placed in the 
White Compartment. | When shock [same intensity as used in the punish- 
Ment demonstration | was applied, rat churned its feet, squealed, singled 
nd of 6 seconds, escaped the shock 


abour in the compartment, and, at the ¢ 
tment. But very soon the rat 


'" running. through door into black compar bel 1 had 
Came back into the white compartment [see comments below i ^is ii to 
e driven our with a second shock. This time the escape took less than 3 


seconds. . , 
2nd trials When, after an interval of 2 minutes, rat was removed from 
bliek compartment and placed in white ipa age x e pm 
Of the white compartment into the black one. No shock applied. A little 
Ater the rat came and looked through the door, into the white compart- 
Ment, but it di nter 
` it did not enter. " SED 
rd, ah ox " «tb trials: Fled from the white compartment into the black 
s b. and geb als: 
One within a second or two, 
Content to stay in the black com 


without application of shock. Rat is now 
partment, some distance from the door. 


demonstration ended. 


In some ways this demonstration w ould have been notera nrag mal 
i after the rat ran into the black com- 


^ the doorway had. been closed ‘ 
venting its return to the white compartment, How- 
cc > á 


Partment, thus pre à 
conducted, revealed a side effect 


Sver, the demonstration, as actually i hi 3 
Often noted in connection with punishment, namely, something w hich, 


can be called the “curiosity” effect. One 
of the students ik dhe semiar where the foregoing demonstration was 
" i 
Made volunteered the information that he had once had a pet raccoon 
SUR : 

‘hich was occasionally 


for want of a better term, 


allowed to roam about in his family's summer 
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cottage. On one occasion, the raccoon found a lamp cord on the floor 
ud serte gnawing into it. Shortly there was a tlash of tire that caused 
the raccoon to bound aw av. Bur, almost immediately, the animal re- 
turned to the cord and started gnaw ing again. However, the second 
occurrence of a short circuit was enough for him, and he thereafter 
eroe d respectful distance. from lamp cords. As will be observed in 
more detail later (Chapter 6). slight fear (of uncertain origin) may 
make an organism explore rather than avoid the stimuli or objects to 
which the fear is attached. Why this should be so has not been fully 
determined, But this fact in no w av goes against the assumption that 
shock or other painful stimulation produces fear and that, in. general, 
fear then makes the organism avoid the stimuli to which the fear is 
conditioned. 

In a sicuation such as the demonstration just described, there is a 

tendency to say that the subject has learned to avoid the shoek—hence 
the term, "avoidance" learning. But this is a rather inexact, abbreviated 
way of speaking. Morc precisely, what the rat has learned is (a) to 
be afraid in and of the white compartment and (b) to reduce the fear 
(and shock, when it is presented) by sanning into the black compart- 
ment. Strictly speaking, it is not the avoidance of shock that is reward- 
ing to the animal and keeps the running response going. It is rather 
the fact that the white compartment arouses fear and the running pro- 
vides a solution to, or escape from, this "problem" or drive. The 
avoidance of the shock is a sort of by-product -tħough, to be sure, a 
very important one. Action which thus appears to be teleological or 
“purposeful ’—and which, in a sense, indeed is—can in this wavy be 
accounted for in a purely causal, or consequential, wav. We do not 
have to say that the rat runs “in order to" avoid the shock; we can 
sav instead that the rat runs because (or by-cause) of fear (see also 
Section Nb). i 


: Now some readers may, at this point, ask: “But why introduce fear 
into the picture? A scientific hypothesis or explanation should always 
be as parsimonious as possible. And the most parsimonious explanation 
In the foregoing Situation is to say, quite simply. that flight from the 
white compartment is a reflexive reaction to the shock which then, by 
classical P: 1 


hes avlovian principles, becomes connected to the stimulus con- 
stell: rovide j i PESA 1y 
elation: provided by the compartment. itself, without shock. Thus. 
the flight reaction which later 
being put into the 
int 


occurs immediately upon the animal's 
: white compartment (before shock is applied) is 
ats Pre asa straightforw ard conditioned reflex, without any refer- 
ence to fear or similar subjective phenomena.” ‘This objection is 
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entirely valid, as far as this particular situation is concerned: here, from 
the standpoint of sheer logic. it is indeed wor necessary to introduce 
Bur the difficulty is that the classical 
range of other 
It 


fear as an intervening variable. 
Pavlovian theory, w hile adequate in this and in a narrow 
us an explanation of broad applicability 
a satisfactory. understanding of passive 
and there are various other 


Situations, does not eive 
does not, for example. provide 
avoidance learning. (ordinary punishment); 
situations wherein it br ks down completely. This is why (see also 


Sections V and. VID) it is incumbent upon us to introduce, at this early 
rather than knowingly limit 


gh valid in this par- 


juncture, concepts with high generality, 
A f explanation which, thou 
ticular situation—and, to be sure, simpler—would, however, soon stifle 
inclusive, pow erful explanatory system. 

in the situation under discussion, the 
less concomitant responses, Which 
action of running (along with 


ourselves to a type o 


the development of a more 
So our assumption will be that. 
Shock initially produces two more or 


are radically different: (a) the escape re 
a number of other similar but ineffective reactions) and (b) the emo- 
tional reaction of fear. In principle. of course, cither of these reactions 
would be conditionable. according to classical Pavlovian theory; and 
Most Pavlovians would simply ignore (or deny) the fear reaction and 
assume, instead, that the running reaction becomes directly conditioned 
to the white compartment (or “CS.” since it “precedes” the shock). 
Bur, as will be shown later, this interpretation. W hile appealingly simple. 
gets us into serious difficulties in other situations; and in light of these 


difficulties we shall make the very different assumption that it is not the 
vr that gets conditioned in this situation 


omic reaction of fear. In fact, it will be 
"voluntarv 7). responses in gen- 
ind that the 


overt, muscular action of runnit 
auton 
avioral ( 
he strict sense of the term, 
to emotional responses (rather 


bur instead the emotional, 
Our premise that ordinary beh 
int 


eral are not conditionable. 


concept of conditioning applies only 


Il later— Chapter >. Section IN; see also 
- have occasion. to. modify this 


Chapter 5 
but for the present it can stand in this un- 


broadly conceived). (We sha 
Mowrer, 1960, especially 
statement ina special ways 
qualified form.) 


IV. Active and Passive Avoidance 
Learning Compared 


The foregoing demonstration helps onc to see the similarity between 
the principles involved! in so-called punishment and what is commonly, 
known as avoidance learning. In both in- 


though also too restrictivelv. 
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stances, there is fear conditioning; and in both instances a way of be- 
having is found which eliminates or controls the fear. The only im- 
portant distinction, it seems, is that the stimuli to which the fear gets 
connected are different. In so-called punishment, these stimuli are pro- 
duced by (correlated w ith) the behavior, or response, which we wish 
to block; whereas, in so-called avoidance learning, the fear-arousing 
stimuli are not response-produced—they are, so to siy, extrinsic rather 
than intrinsic, independent rather than response-dependent. But in both 
cases there is avoidance and in both cases there is its antithesis, punish- 
ment: hence the impropriety of referring to the one as “punishment” 
and to the other as “avoidance learning.” Obviously precision and 
clarity of understanding are better served by the alternative terms here 
suggested, namely, passive avoidance learning and active avoidance 
learning, respectively, 

Perhaps the relationship. and also the distinction, between these two 
forms of learning is best expressed by noting that in active avoidance 
learning, there is first fear (produced by some external stimulus, or 
danger signal) and then overt: response (which the fear produces), 
Whereas in passive avoidance learning, there is first a response ( produced 
by some other drive, such as hunger) and then fear produced by re- 
sponse-correlated. stimuli). What the organism has to do, in the two 
cases, to reduce the fear is, of course, quite different, because of the 
difference in the nature of the stimuli to which the fear is conditioned: 
activity which eliminates (or removes the subject from) the external 
stimulus, in the one case, and cessation of the activity s hich is producing 
the response-correlated stimuli, in the other. But, as we have seen, the 
two phenomena involve exactly the same basic principles, of fear con- 
ditioning and of the reinforcement of whatever action (or inaction) 
eliminates the fear. 


A student, after watching demonstrations of both these forms of 
avoidance learning, was once 


an ' g prompted to observe: "In passive avoid- 
MAES learning the subject is responsible for what happens, while in 
active avoidance learning he is not. In the former situation, he gets into 
trouble only if be does something wrong, whereas in the latter, the 
ie Mun Higthlews of what he docs." Intuitively, one senses 
Ru Noon ident had in mind here. Yet there is a sense in which even 
she i apt: onc could say thar in active avoid- 
done, whereas in nee pew pd i something the subject has ef 
the subject bas die, In as ree eae because of something 
say, a way of enorct sn e avoidance learning, the shock is, so to 

* g or ensuring an action, rather than preventing 
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one. So the subject is "responsible" for. or has control over, what hap- 
cases in the sense that he can determine whether shock 


pens in both 
by making or by inhibiting some specific re- 


will or will not occur. 
sponse.” 

But a ditference in “responsibility” 
nature of the conditioned stimulus or 
ance the subject himself “makes” the stimuli w hich are associated. with 
shock, whereas in active avoidance these stimuli are made by some 
tor by “nature”). In the latter case, the danger signal 
“Do this (or that), or pain will follow— 
the mood is conditional: “If 


does exist with respect to the 
danger signal: in passive avoid- 


other organism 
is in the "imperative" mood: 
so get along?" But in passive avoidance 
vou do this (or that). pain will follow—so w atch out." 
ontrast between active and p: 
vet another wav. By means of a noxious 
ak. one can teach a rat (or other organ- 
h to do or not to do some- 


sive. avoidance 


The similarity and € 
learning can be indicated in 


stimulus, such as electric shock. 
g it does not wis 
both situations imply conflict; 


obvious in passive avoidance, 


ism) either to do somethin 
thing it wishes to do. In other words, 
on is usually more 
because of hunger 
abe blocked only if a competing motive 
ay at first seem to be 
a against move- 


but this. phenomen ve c 
where the subject, or some other definite drive, is 
set to do something, which cat 
3 avoidance, there m 
is always some iner 
at least this to contend with when 


anism to do some- 


(fear) is introduced. In active 


no competing motive; but there 
and there is 
ce, or drive, an org 
rom doing something. Hence, al- 


ment (see Chapter 11); 
an attempt is being made to cock 
thing, as opposed to preventing it 1 
though conflict is not v holly absent in active avoidance (and may be 
ma fatigued or ill organism), in passive avoidance 
definition—is 
underlying the 


relatively intense, 
contlict—almost by 
proportional to the drive 


always present and of an intensity 
behavior which is being 


blocked. 4 a 
Somewhat parenthetically it may be remarked that the foregoing Is a 
fair exhibit of the integrating OY "resolving" power of the type of theory 


systematically explored in this book. Many similar instances will be dis- 


covered as we proceed. 


will be discussed in more detail later in 


(1954). W hicl 
is made thus: In active avoidance, only 


In a paper by Dinsmoor 
this chapter, the distinction. " 
ded (by termination of tl 
in the sense t 
is punisbed and all others are. "rewarded," 


in paraphrase, 
p signal and reduction of 


ne danger 
hat they fail to avert shock; and 


One response is rew 
fear) and all others are “punished, 
in passive learning, only one response 


t. of being safe and therefore in welcome contrast to the 


in the sense, at lea 


Specifically punished response 
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V. More Complex Instances of Fear, and 
Fear-Motivated, Learning 


Though the practical differences between passive and active avoid- 
ance learning are important ones, we thus sce that the theoretical dis- 
tinctions are trivial: Both involve the conditioning of fear to specifie 
stimuli or stimulus compounds, and the organism then behaves in a w ay 


calculated to reduce or control that fear. In the one case, the subject 


learns what vot to do and in the other case what to do. Also, we see 
how unsatisfactory is the term, “avoidance learning," when used to 
imply a contrast with “punishment.” In both Cases there is an clement 
of avoidance, pain avoidance; and in both, there is a possibility of pain 
infliction. Moreover, some measure of conflict is always present in both 


types of situation, i ; 

The procedure for demonstrating active. avoidance learning. with 
rats which has been described in the preceding section was developed 
and first reported by N, E. Miller. (19482). | his investigator also cx- 
tended the procedure in a number of ingenious w avs, showing that 
once a rat has been conditioned to fear one compartment of the ap- 
paratus, it can be made to learn fairly complicated ways of getting out 
of that compartment and into the other, safer onc. For example, if the 
opening between the two compartments is obstructed by a sliding 
door, the rat will learn to press a little bar located nearby or perform 
any of a variety of other "instrumental acts” if they. cause the door to 
open and thus permit entry. into the black compartment. On the basis 
of this and similar types of experiments, fear has been termed an 
acquired, or secondary, motive or drive. Once acquired, through con- 
ditioning, it evidently acts very much as do such primary motives as 
hunger, thirst. and cold, to "drive" the organism until a means of re- 
ducing or eliminating the motivational state is found. 

It is also possible to exemplify: 
way. 


stive avoidance learning in another 
In the Miller-type demonstration, fear becomes connected to a 
complex of stimuli, ie., to a situation, Fear can also be conditioned to a 
more delimited stimulus (Mow rcr. 
l.ct us suppose that the appar: 
modified, by 


19503; sce also Chapters 3 and 4). 
atus described in the preceding section is 
Temoving the partition between the two halves, 


) | painting 
the whole interior the same color, 


c and changing the floor so that shock 
can be administered on either side. Let us suppose, also, that there is a 
buzzer on or near the apparatus which c 


an be sounded by the experi- 
menter. The procedure will then be $ 


as follows. A rat is put into the 
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apparatus and allowed to explore for 10 minutes or so. Then the 
buzzer, which initially produces only a momentary, "listening" reac- 
tion, is sounded for five seconds and the shock is applied to that half 
of the grill floor on w hich the animal happens to be located. As in the 
rst not "know" specifically 


preceding demonstration, the rat will at fi 
what to do about the shock. Soon, however, it will escape by running 
to the Opposite (uncharged ) side of the apparatus, at w hich point the 
buzzer will also be turned off. At intervals of 3 
buzzer and shock will be repeated, until the animal has had a total of, 
let us say. ten trials. The rat will then be returned to its living cage 
when it will again be taken 


yout TWO minutes, the 


and allowed to “rest” until the next day, 
another set of ten trials. This procedure 


to the apparatus and given 
arned to run to the opposite end 


will be continued until the rat has le 
zer alone. 
a rat may make one or two 


of the apparatus in response to the bu 
Under the circumstances just described, 
he buzzer on the first day, or it may make none 


runs in response to d 
almost certainly. begin to 


at all. However, shock-avoidance runs will 
and by the sixth or seventh dav the 


h runs out of a total ten trials. 
als in all, most animals will be 
runs to the buzzer alone 


occur on the second or third day, 
animal may be making six or cight suc 
By the end of 10 days, that is, 100 tri 
making 9 out of io; oF even 10 out of 10, 


(see Fig, 2-2). 
a demonstration of this 


thing one observes in 
of what may be called 


and disappearance. 
About the time the animal makes 
alone, it will be scen occasionally 
other, between trials. 
istinguished between 


Another interesting 
kind is the appearance, 
“spontancous,” or “interval.” runs. 
Its first runs in response to the buzzer 
to move from one side of the apparatus to the 
animal has not yet « 
1 situation-without-buzzer. It is the former 
ind as soon as the animal learns this, the 


running occurs quite specifically 


This is a sign. that. the 
Situation-with-buzzer ane 


and it alone that is dangerous, 
Interval responses drop out and the 


to the buzze 

Watching the foregoin 
learning is a good deal slower tha 
experiment. A better way to put 


g demonstration, one is likely to say that the 
n that which occurs in the Miller-tv pe 
it is to say that the animal has a good 


Responses to situation alone, i.e, without buzzer, disappear because, in à pro- 
perform no function: the next trial comes at 


cedure of the kind described. they 
rdless of whether such “interval” runs do or do not 


The interestin 
ase of postponing the next trial, will be con- 


precisely the same time, re 
tak nae : 

tke place (see Chapter i! 
are allowed to be functional, in the ser 


). 


g situation wherein berw cen-trial runs 


sidered in Chapter i2. 
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deal more to learn in the "shuttling" situation. The rat not only has to 
learn what to do about the shock and then about the buzzer; it also 
has to learn, as already noted, that the situation is dangerous only when 
the buzzer is present, In the beginning, the whole apparatus, being 
associated with shock presentation, becomes fear arousing, and before 
the animals behavior can become smoothly adaptive and efficient, fear 
of the situation when buzzer is absent has to extinguish (Chapter 11). 
In other words, as already noted, the animal has to form a discrimina- 
tion (see Chapter 12) between. situation-with-buzzer and situation- 
without-buzzer. Because many so-called conditioning experiments have 
called for this type of learning, over and beyond the conditioning 
proper, “conditioning” has commonly. appeared to be a more difficult 
and slower form of learning than it actually is. If the solution to the 
problem of acquired fear is kept simple, as it is in the Miller experi- 
ment, we see that the "conditioning" occurs quite rapidly, in a trial 
or two, instead of requiring upwards of a hundred trials, as is likely 


| 
i 
3 
s 5l 
B ^| 
= 1| 
1| 
=— " : - n j 
10 0) 30 40 50 60 70 80 3X) 100 


Trials by blocks of 10 


Fig. 2-2. Typical performance of a laboratory rat 
ing. While comparing favorably ^ 
mental” conditioning h 


in shuttle-box avoidance learn- 
with many other reported instances of. "instru- 


« this learning is obviously rather slow and does not show 
pure form, and ar its best. 
UND). simple escape learning may require 


conditioning in : i s 
S Under optimal circumstances (see 


only two or three trials. Ihe far 
shuttle-bos and similar experimental pro- 
conditioning is confounded with a high 


Hence, the apparent slowness of “conditioning” 
of the subject. 


greater number of trials required in the 

cedures is due to the fact that here 

order of discrimination learning. 
and the apparent "stupidity" « 
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where, in addition, a discrimination of some sort must also be formed 
(Mowrer & Lamoreaux, 1931). 

In a demonstration of “shuttling” of the kind just described, the 
follow ing stages or levels of learning must occur: 


i The rat has to learn what to do about the shock—how to get it 


turned off, how to escape from it. 
>. Because shock is accompanied and 


the buzzer alone (see below) becomes 
or secondary, problem or drive to deal 


slightly preceded by a buzzer, 
B 


capable of arousing fe 


3. The rat now has a second, 
With, namely, the fear which is now conditioned to the buzzer. In 
favorable situations, the subject now l 
escaping from it (and, in the process, avoiding shock). 

4. d However, for a considerable time the rat fails to discriminate 
and situation-w ithout-buzzer and runs some- 


earns W hat to do as a means of 


between. situation-with- 
" Onlv when the subject has reached. the point 


What “indiscriminately. 
is “avoidance learning said to be clearly 


of running to buzzer only 
and unambiguously: displayed. 

is here a good deal more com- 
ble forms of fear conditioning 
the Miller demonstration), all 
r with a given place (situation, 
yglv; whereas, in the shuttle- 
ically: connected with a 


Manifestly, the total learning task 
plicated than it is in the simplest possi 
and escape. In the latter. (for example 
associate fe: 


the organism has to do Is to 
à ex)—and to act accordir 
fear has to become spec 
m the particular situation in which 
It is useful to keep this difference 


or stimulus comp! 


box procedure, the 
less dissociated fre 


cuc, and more or 
the cue occurs (see Section. V 1). 
clearly in mind. 

VI. Generalization and Discrimination, 

Preliminary Considerations 

learning ultimately exhibited in the situation just 


Although the ys pie 
described is a good deal more specific, and more difficult, than. that 
> Miller demonstration, 
an be illustrated in the following way. Let us sup- 


rat to "shuttle" as a means of escapine 


tue : 1 )0, is somew hy ituation- 
Meglued in the it, too, is somewhat situation 


bound, This point € 
pose that instead of teachin 


from: the shack and, ther after, of terminating the fear-producing 


buzzer, we decide to teach it to le 
ready seen, is a response Ww hich a rat is naturally disposed to make to 


and if this response is rew arded by drive reduction, 


ap into the air. This, as we have al- 


electric shock; 


38 LEARNING THEORY AND BEHAVIOR 


it will occur more and more promptly, first to the primary drive of 
shock and, then, to the secondary drive of fear, as aroused by situation- 
plus- buzzer. 

After our subject has learned to jump in the manner described—and 
with a high degree of reliability—to the buzzer under these conditions, 
let us do “the following. We take the rat out of the apparatus and put 
it on a table. Then, after the rat has had an opportunity to explore for 
a while, we sound the buzzer. The chances are very high that the rat 
will now vot leap into the air. ‘The reason is that the conditioning of 
fear to the buzzer has been jg a particular situation; and when the 
specific buzzer stimulus is Lors under altered conditions, there is 
comparatively little spread, or generalization (scc C hapter 12). of the 
fear—thus showing that the fear has not become conditioned to the 
buzzer-as-such but to buzzer-in-a- given-situation, 

The fact that a buzzer is a danger signal when heard in the box- like 
apparatus does not necessarily mean that it is dangerous, ic, will be 


followed by shock, under all circumstances. As a consequence, the 
nervous systems of living organisms have evolved in such a way as to 
Sar them from ov er-gener alizing. In other words, conditioning tends 

> be specific to the situation in which it has occurred; and if the 
situation changes, the tendency for the conditioned reaction to occur, 
even though “zbe conditioned. stimulus” remains the same, is con- 
siderably reduced. Hence, a rat that has been conditioned to buzzer- 
in-box will not react with the same degree of fear to buzzer-without- 
box. What this demonstration shows, ‘therefore, is that although the 
shuttling or jumping response as it occurs in the apparatus scems to be 
caused quite specifically by the buzzer, “the conditioned. stimulus 
is really buzzer-in-a- particular-situation.! 

A bright student, upon hearing the foregoing demonstration. de- 
scribed, called attention to an inconsistency bern cen it and the story 
of the boy (Section 1) who taught the cats to stay away from his 
plate when he coughed, by associating cough with * punishment? in 
another situation, W hile valid in principle, the story is clearly at fault 


‘Parlier, reference has been made to the tendency which rats show, in the 
shuttle-box experiment, to make the running response in the interval between 
trials, i.e., the tendeney for their fear to generalize from box-with-buzzer to box- 
without-buzzer. In the situation just described we tested to see to what extent à 
rat trained to react in a given way to buzzer-with-box would generalize to buzzer- 
without-Pov. As noted, generalization of the latter sort is less marked, suggesting 
that conditioning, without spec be 


al training (see next paragr: 


aphy. tends to be more 
to the situation-as-a-whole than to a particular segment thereof, Les to a particular 
“stimulus.” 
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-than would, in fact, 
T 


ane Neral more generalization of fe: | 
dd Pd fasti What one would have to do. with both the bu 
Part s ULM x fear to generalize M idely. W ould be to “con- 
na diac ore such al stimulus in a variety of different situa- 

. dn a new situation, W here there had been no reinforce- 
| to occur to the specific stimulus, full 


2d the fear might be expectet 
re k pihen words, the “meaning” 
Tite 3 lave to be made abstract, V0. 

sis a state which has to be achiever 


of the buzzer, or the cough, 
independent of the situation. 
] with those stimuli which we 
at all efficiently. Although 


Galli TED A r . 
words" before language can function 
“by the context,” 


Re be | odified, ais wes SIN. yd n : 
a i distinguishing thing about “yvords” is the stability of their 
meanings.” regardless of the situation or circumstances under which 

z and 4). Much the same ob- 


the "oap * 
V are used (Mowrer, 1960. Chapters 3 
| respect to the medium of ex- 


they c : 
N continue to be somew hat m 


servati a i 
ations can, of course, be made witl 


ch; 5 
ange known as money. 


VII 
- Fear as an Intervening Variable 


t simply reverses or stamps out 
nifestly limited in that there is 
derstand what happens when 
vething done, but because of 
ing sections of this chapter, 
assive and active avoidance 
that in both cases a crucial 


AT ena co ; 
x horndike's theory. that punishmen 
Ye learni : cm MET 

learning involved in a “habit Is ma 


Noth] S ‘ 
hing in it which would also help us un 
not because of son 


punishment" is applied. 
e m left undone. As noted it preced 
ar more satisfactory view is that both p 
' and 


the same way 
formerly neutral (response- 


Tease 
ar 3 

ning operate in much 
fear to 


fact is the conditioning of 
at) stimuli. , 
l conception of the way “punishment” (in 
citatory forms) operates Is importantly de- 
itioning, one MAY be inclined to infer 
and that Pavlov had the answer. 
in fact, be valid. Empirically, Thorndike 
ervations as to how punishment (in its 
in regard to the theory, or 


dependent or independer 
eo this improves 
peden E und Ss s. NS 
that i OB the principle of conditit 
Nahe Tornike M as “all wrong 
er assumption would. 


Was enm 5 
S entirely correct in his obs 


inhibi 3 : - 

hibitory guise) operates it was only 

CRD): oi us “4 offere i c was d 

Nplanation, thereof which [Thorndike offered that he was in error. 
rofessed reflexologist and 


a self-p 
alistic as fear; 
v conditionable, in keeping 


was 
tive and ment 
iis readil 
s of fear conditioning made 


Pay 

ay . A . 

ex hk it will be recalled. 

lewed anything so subjec and although 


rricular re 


We fj ~ ; 
3 € find that this pz actor 
M : "im" jcatl 

“ith Pavlovian principles. the application 


fferent from anything for w hich Pavlov 


In i $ Š 
this chapter are quite di 
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himself responsible. His attempt was to found 4 scIenec be b» 
havior which was based exclusively upon unconditioned (native) nf 
conditioned (acquired) “retlexes.” involving manifest movement o 
observable parts of the body and with no “intervening variables. 
Actually, Pavlov did resort to intervening variables, or constructs. 
of a kind. As already noted, he was meticulous in avoiding anv and 
all psychological constructs, such 


as fear obviously is; but he made 
considerable 


use of two purely hypothetical states of the central 
nervous system which he called excitation and iubibition. Under the 
influence of positive reinforcement, Cr, 
dog, “excitation” 
cerebral cortex 


food presented to a hungry 
Was supposedly generated in a localized area of the 
«from which it then radiated out into adjacent. cortical 
area. And, conversely, under the influence of Negative reinforcement, 
ega Weak acid injected into a dog's 
to some other area of the 
in the appropriate 


mouth or clectric shock applied 
body, "inhibition" was supposedly generated 


area of the cortex and, like excitation, radiated out 
Into surrounding parts of the brain, 


From the latter way of thinking 
direct theory of punishment; 
and inhibition has itself never been empirically confirmed, and Te 
little systematic effort has been made to apply it to the problem J 
punishment, (In. fact, Pavlov himself did not make much use of his 
theory in this connection but 
"experimental. neurosis” in 
situations, 


it is possible to derive a kind of in- 
but Pavlow’s notion of cortical excitation 


used it mainly to explain the results of 
his dogs as it was produced by contlict 


Vhe impairment. of. function Which sometimes resulted 
Pavlov attributed to the 


citatory and inhibitory stares in the same or adjoining areas of the 
brain.) The real difficulty, of course, was the fact that Pavlov tried w 
derive an explanation of inhibition and facilitation (or excitation) In 


purely. cortical (central nervous system) terms, without making any 
systematic reference to the autonotic 
which it mediates, one 


simultaneous presence and “clashing” of cx- 


Nervous system and the responses 


of the most important of which is fear (cf. Chap- 
ter 6, especially Section IX), 
system and of the 
for Pavloy 
more 


This neglect of the autonomic nervous 
emotions in general was to be a very costly mistake 
ll shortly see); and it seems all the 


hat most of Pavlovs experimental 


and his school (as w e sha 
remarkable when one recalls t 
work on conditioning Was done with the response of salivation, which is 
itself mediated, nor by the central nery ouss 
nervous system and which ( 
Comitants, 


Ystem but by the autonomic 


as We shall also sec) also has emotional con- 


Inherent in Pavlovs basic Conception of conditionine there is. 
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actually, à somew hat better possibility of explaining punishment alone 
quite ditferent lines. Ler us assume, for example, that a dog has been 
approach reaction to some conditioned 


Gught to make a "positive" or 
the dog comes for food. Now we 


stimulus: when a whistle is sounded 
take the whistle and pair it with a different unconditioned: stimulus, 
he one that produces a different response. Suppose, for example, that 


every time the whistle blows the dog now gets shor with a small B-B 


gun. Flight rather than approach will be produced by this latter 
stimulus, so it would be a natural expectation that the dog would now 
response, rather than with the 


ans of the same simple assump- 
wcorv of conditioning, it is 
of punishment: that is. 


begin to react to the whistle with this 
original one of approach. Thus, by me 
tions which constituted Pavlov's general d 
it least an oblique theory 


one which suggests how it would be possible to inhibit or reverse 
Bur at once we see that the example 


as we ordinarily think of it; and 
c Pavlovian approach is. generally 


possible to derive : 


previously conditioned behavior. 
given does not typify punishment 


other considerations show that th 


Unsatisfactory. ts 
realization that whereas Thorndikian learn- 


Vhus we arrive at the 1 i 

ount (at least formally) tor punishment, it could 
| for active avoidance learning; and whereas 
Pavlovian theory could (again. formally) account for active avoidance 
learning (see last paragraph of Section I), it was. as we have just seen, 
uite unable to handle the problem of 
schemes inadeq 


when we 
vorndike's views in this area, 


Ing theory could acc 
Sive no explanation at al 


punishment. ‘Therefore, we find 
in important, though dif- 


look more closely at even 


cach of these two major WAKE 
ferent, ways. And, it so happens. 
t z s of Pavlov's and TI 
t isfac 
feedback theory of punishment 
deal more plausible than Thorn- 


ina position to show 


he strongest feature 
ley, too, are consider: 


ably short of sat tory. In preceding sections 


it has been shown that the conflict or 
Gnvolvine conditioned fear) isa good 
n and we are now 


really competent account 


ikes stamping-out hypothesis; 
wise does t 
In other 


or give i 
words, on close inspection, it 
Pavlov's theory. as originally 


= 


vat Pavlovian theory like 


Of active avoidance learning. 
‘Vhorndike’s nor 


turns out that neither 1 
xplanatior 


formulated, a first-rate ¢ 
avoidance learning. 

: So we now turn to amore 
Ing, which superficially seem 
Conditioning interpretation bu 
embarrassment to that theory- 


1 of either active or passive 


1 give 
detailed analysis of active avoidance learn- 
ed to lend itself so re 
t which, in reality, was to prove a great 


lily to a classical 
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VIII. The Classical versus Instrumental 
Conditioning Dilemma 


Although it was Pavlov who gave the concept of conditioning W idest 
scientific curreney and who tried to make it explain all behavior modi- 
fication, his own researches, as already noted, were very largely re- 
stricted to the conditioning of the salivary response; and although 
salivation has both an appetitive and an aversive or avoidance aspect 


(actually different glands are involved), it was another Russian phys- 
iologist, VM. Bekhterev (1913). who pioneered the inv estigation of 
overt avoidance behavior, from the conditioned-retlex standpoint, This 
investigator found, with dogs as subjects, that if a buzzer or some other, 
originally neutral stimulus precedes an electric shock delis ered, for ex- 
ample, to a dog's paw, the dog will at first lift his leg only to the shock 
but, after a few pairings of buzzer and shock 
leg “defensively.” to the buzzer alone. Supertic 
avoidance lcarning looks like a pure instance of 
simple stimulus substitution. 
shock, is thought of 
made to the buzzer, 


will then start lifting the 


Iv, this exhibit of active 


retlex conditioning, or 
If the leg flexion, made in response to 
as an unconditioned reflex, then the same response, 
can be called a conditioned reflex, without refer- 
ence to any sort of intervening, subjective factor such as fear. But it 
Was soon to be discovered that even in this seemingly perfect example 
of classical avoidance conditioning, there was a serious flaw. 
According to Pavlovian theory, learning is contingent upon the tem- 
poral conjunction, or contiguity, of a formerly neutral. stimulus and 
one which, from the outset. is capable of producing the desired re- 
action. "hus, in the Bekhterev type of experiment, the conditioning 
of leg flexion is dependent upon the pairing of buzzer and shock; 
and, per theory, the more frequently these stimuli are conjoined the 
better will be the learning. But it is now well known that, in a 
situation of this kind, better learning is usually obtained if the CS 
(conditioned stimulus) is followed by the US (or UnCS, for uncondi- 
tioned stimulus) only in the event t 
in the expected way to the CS 
the UnCS is omitted, so th 
in that the conditioned resp 
unconditioned stimulus. 
procedure of invari 
Hunter (1935), and 
of the first Americ 


nt that the subject does sot respond 
alone. If such a reaction does occur. 
at the procedure is said to be “instrumental” 
)onse prevents the occurrence of the noxious 
This is to be contrasted with the "classical" 
ably pairing CS and Cts, Schlosberg (1934). 

Brogden, Lipman, & Culler (1938) were some 


an psychologists to work on this problem; and the 


43 


EFFECT, CONDITIONING, AND PROBLEM OF PUNISHMENT 


experiment by the last-named investigators gave especially dramatic 
and striking results. This experiment has been summarized by Hilgard 


& Marquis (1940) thus: 


A guinea pig was placed in a [ revolvable] cage and after a conditioned 
stimulus (buzzer) it was given a shock which evoked running behavior. 
One group of animals, trained according to the Pavlov procedure, were 
shocked whether or not they ran. Another group. trained according to the 
learning, were not shocked if they ran. Learn- 
Ds but reached a much higher level of 
The results are plotted in | Fig. 2-3]. 
r not they ran continued to 


arrangements of avoidance 
Ing began similarly in both groups 
performance in the second group. 

Guinea. pigs which were shocked whether or ^ 
show anticipatory agitation at the sound of the buzzer, but after the first 
few trials the tendency to run did not increase; those shocked only if they 
did not run developed the habit of running promptly at the sound of the 
buzzer, Learning in this situation appears to be based in a real sense on the 


avoidance of the shock (p. 58). 
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— —- No avoidance 


Fi 4 
19. 2-3. learning curves, re- 
P poneed from Brogden, 
Apman, & Culler (1938. P- 
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€ te 
onflict generated by the 


no i 
Navoidance procedur 


ditioning some response. initially clicita- 
1 becomes capable of being 
ineffective st s a consequence of the 
al contiguity,” of these two stimuli. 
was termed “reinforce- 


Dass M ^ 
Pavlov had held that in con 


ble only by an unconditioned stimulus, 


imulus, 


licited by: some formerly 
"tempor: 


of CS and UnCs 
“reinforcement occurred. the 


paired presentation, in 
Vhis paired presentation 
Ment.” and the more frequently such "rei 
for the CS to clicit the response in question 
s of the Brogden-Lipman-Culler experi- 
see pp. 42 ff. and the two 


str et 
longer the tendency 

x ue "e 
Upposedly became. The result 


Ment (and of various other investigations. fcr iav Te 
following chapters). brought this assumption under grave suspicion. 
: E Apre (invariable) pairing of CS and UnCS leading 


"Or n p 
r here we see frequent 
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to extremely poor learning. and decidedly less frequent pairing lead- 
ing to extremely good learning. 

‘In the passage quoted, Hilgard & Marquis remark that, “Learning 
in this situation appears to be based in a real sense on the avoidance of 


the shoc 
learning, in this situation, involves two ditferent phenomena: (1) the 


We now can be a good deal more explicit and sav that 


process W hereby the subject learns to be afraid when the buzzer sounds 
and (2) the process ww hereby the subject learns w hat to do about its 
fear. The first is, to be sure, dependent upon the paired occurrence of 
buzzer and shock; but once this learning has occurred, it is obviously 
wise to omit further CS-CnCs pairings on those trials when the 
subject responds to its fear by running (since this is the response 
which we supposedly ish the subject to acquire). For, as already 
noted, if we vow pair CS and UnCS we are creating conditions which 
> punish the subject. for running and thus set up a conflict: 


effective 
between fear of the buzzer, w hich tends to evoke running, and fear 
of the stimuli which running produces, which tends to inhibit running. 
That the so-called instrumental conditioning procedure should give 
better results in active avoidance learning than docs a noninstrumental 
(or "classical") procedure not only follows from considerations ad- 
vanced in carlier sections of this chapter: it is the sheerest of com- 
mon sense. In fact, the “classical” procedure, wherein the subject is 
"punished" regardless of whether or not he does what he is "told" (by 
the CS) to do, would seem no less absurd and "unfair" than would 
a passive "avoidance" learning procedure in which the subject is 
punished not only when he mak 


1 n "forbidden" response but 
also when he, quite obediently. refrains from making the response.” 

Recently Razran (1936€) has pointed out that the superiority of the 
instrumental. conditioning procedure, over the classical procedure, 
was known in Russia considerably before it was discovered in this 
country. He says: i 


Both avoidant and unavoidant conditioning were studied in Bekhterev’s 
"association-reflex" laboratory from its beginning in 1907. Protopopov ( 1909) 
in his pioneer experiment on shock-conditioning used unavoidable shock w ith 
five dogs, whereas Molotkov (1910) in conditioning four human adults used 
avoidable shock. ... i 

In 1926, Starytzin introduced avoidable shock with dogs. He compared 
the conditionability of dogs under avoidable and unavoidable shock. His 
results antedate those of Schiosberg’s (1934) « ith rats by eight. vears. . 6 + 
Srarvtzin found avoidable shock superior to wnavüuidublereua — 

Starytzin’s procedure was modified and perfected by Petropavlovsky 

For further criticisms, of a different kind, of classical conditioning method- 
ology. sec T iddell rygg? j 
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(1934). and since then Russian experiments have been using both avoidable 


and unavoidable shock with dogs and other animals (p. 127)- 

f allowing this finding to have the stimulating 
and research) which it had in this country, the 
in such a manner as not to 


However, instead o 


etleet Con both theory 
to explain it away, 


E 
Russians have attempted 
This is made clear in Razran's 


disturb official. Pavlovian doctrine. 


further comment: 

have as a rule found avoidable shock 
alized it as a distinct form of 
shock began in this country 
anything, inferior to un- 


Despite the fact that the Russians | 
mare efficacius, they have nor eoncepru 
conditioning. Conceptualization of avoidable 


With Schlosberg, who found avoidable shock, if 
All the Russians seem to say about the superiority of 


it is “more natural," more "in line. with 
Irs from pari | reinforcement. [see footnote 


intorcement" (pp. 1277! 28). 


avoidable shock 
avoidant conditioning is that 


animal evolution,” that it rest 
“non-overt 


Oj or, more exactly, from 
orcement" is clearly ad boc and im- 
| known facts. For example, if the 
ssical conditioning procedure were 
* the logical way to improve 
srocedure but one in which 
every third trial, or 


of "over-reinfi 


The concept 
are witl 


provised, and does not squ 
oduced by the cla 
"reinforcement." 
an instrumental | 


other trial, or 
subjects own behavior. 


inferior results pr 
indeed due to toa much 
the situation would not be 
shock followed signal on every 


every. fourth trial, without regard for the 
ritted (fixed-ratio) reinforcement would, presum- 


inforcement,” but it would still not produce nearly 
which are obtained where the shock occurs 
Is to respond to the signal and is withheld 


his type of intern 
ably, avoid "over-re 
as good results as those 
only when the animal fai 


When the animal makes the ¢ There is nothing in the 


asc. 


xpected respor 
count for the superiority 


t which can ac 
al procedur 
inuous) reinforcement. 


Concept of over-reinforcemen : 
c over those of a pre- 


of the results of a truly instrument 


arranged. schedule of intermittent. (07 cont 
ent has commonly been found to result in greater 
o extinction, 
argued that the so-called 


Since intermittent reinforce : 
than does. continuous 


habit strength,” as measured. by 
12). 


resistance f 
etimes been 


it has som à 
results. simply and precisely 


reinforcement (see Chapter 
instrumental conditioning procedure gives better 
intermittent reinforcement. 
among them these: 


results. in better. acquisition, 
al procedure are most apparent (2) As suggested 
] without reference to M ise 


bec 1 phere are sev eral reasons for 
cause jt ir z » 
wolves 4 s E 
ci) There is no ev idence that 


doubti t. J 

; ubting this interpretatione ind this is where t 

ntermi S E aC zpe CERE 
ermittent reinforcement 


ads A d 
antages of the instru" 


yent 
inforceme 
trial, almost 
And (3) there 
greater resistance to extinc- 


ammeu 
certainly will not give the same 
is current evidence for 


above, intermittency of re nt, progr 
the subject does on a particular 

results as will instrumental reinforcement 
reinforcement 
“habit strength, 


holima cosülts 3 
. 4 : esults in 
teving that intermittent IO 
but because such a training. pro- 
reater s 


tie 
M, not because of g 
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It has already been pointed out that the resolution of this and other 
dilemmas lies well within the scope of conditioning theory provided 
the theory is revised in certain crucial ways. If we assume that, in both 
active and passive avoidance learning, what happens, first and foremost, 
is that fear gets attached to a formerly neutral stimulus or stimulus 
complex, then all follows naturally enough, One can thus explain the 
active avoidance which occurs when fear is aroused by an independent 
stimulus; one can explain the passive avoidance which occurs when 
fear is aroused by response-produced stimuli; and one can explain the 
rather preposterous conflict which results from the use of a classical 
conditioning procedure, where fear gets attached both to an independ- 
ent stimulus (Oks buzzer), which says do, and to response-produced 
(e.g, proprioceptive, ete.) stimuli, which says do not! 

Thus, somewhat paradoxically, we find that by restricting the con- 
cept of conditioning, we greatly increase its applicability and power. 
The reflexologists hoped, as we know, to account for all behavior in 
terms of nativ 


and acquired (conditioned) reflexes. But in thus being 
so ambitious for their principle, they actus 


lly robbed it of its true im- 
portance and value. In fact, thev made a travesty of the concept. And 
what we are here trying to do is to show how, by limiting the concept 
of conditioning to emotional (autonomic) learning, we can develop 
an explanatory system of remarkable scope and consistency. 


IX. Theories of Punishment and Common Sense 


A recent technical paper by Dinsmoor (1954) on the psychology of 
punishment begins with these words: 


A possible reason for the seeming neglect of the topie of punishment in 
contemporary behavioral research and in most of our handbook and text- 
book presentations may be found in the present entanglement of theoretical 
treatments. So confused is the current picture that Stone (1950), in d . 
review of the literature, was led to remark that “The task of resolving ap- 
parently conflicting results . . . is an all but impossible one" (p. 34). 

Dinsmoor then goes on to say: 


Actually... I feel that there is an available formulation which can 
handle the bulk of the data and which can incorporate it within a more 
general descriptive framework without requiring new e 


T. xplanatory princi- 
ples (p. 34). ^ 


cedure reduces the effectiveness of the extinction process (Chapter 11). Thus, the 


attempt to account for the superiority of the instrumental conditioning procedure 
in a way which would by-pass the necessity of fear as \ 


3 : an intervening variable falls 
short of its objective and will not be further considere 


d (cf. Bitterman, 1957). 
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And the discussion which follows is v 
explored in the preceding sections of this chapter (with certain cx- 
ceptions w hich will be considered in Section Na). Few would today 


rv much along the lines 


deny that our understanding of both active and passive avoidance 
is much more svstematic and unified than it was only a few 


. But the remarkable thing is that, even now, our knowledge 


learning 
Nears ago 
in this area cannot be said to represent much of an advance over com- 
o ever drove a team of horses, or a voke of 
knowledge of all the principles reviewed 
progress within the confines of 


mon sense; anvone wh 
oxen, had at least a working 


in this chapter. What appears to be 
therefore, little more than a recovery from 


scientific psychology is, à 
e field was thrown by dramatic, but in- 


the confusion into which th 
adequate, carlier formulations. 

As already noted (Chapter 
captivated by the idea that the 
Was to base it upon observable 


1), Thorndike and Pavlov were both 
way to have a true science of behavior 
correlations between stimulation and 
this notion they both advanced ap- 
ise quite different) suggestions as to 
rning) takes place. "The trouble was 


Overt response; and in pursuit of 
pealingly simple (though otherw 
how behavioral modification (lea 
that their formulations, in striving for parsimony and objectivity, over- 
shot the mark, and resulted in co 
"iconic" with respect to the 
oversimplification, one is tempted to con- 
ave been so different!) And so strong was 
| the turn of the century, with the ob- 


neeptual models which were much 
too simple to be actual. phenomena in 
question. (Without this 
jecture, they could. not h 
existing. dissatisfaction, aroun 


ay of explanatory concepts and categories 


vious prolixity and disat 
any promise 
a modicum of ct 


of simplification and unification, if 


in psychology that TAa Ne 
E npirical justification, Was sure 


made with boldness and 
to have a favorable reception. 

always aims at sim 
‘by these efforts to reduce explanatory prin- 
ty. The difficulty is, they have not been 
nty-five years has been devoted, in 


Science, of course, plicity of explanation; and no 
great harm has been done 
ciples to the barest necessity. 
sufficient: and much of the last twe dc 
psychology, to the task of revising, expanding, and uniting these 
formulations. Today we have re 
ds connection; 
sters, examine other developments 


ason to be well pleased with what has 


been accomplished in d and on this now presumably 


firm foundation we shall, in later € n 
Which take us from the so-called negative to the more positive aspects 
? Psychologists who studied w ith Thorndike, especially in the early and middle 
stages of his career, say that he often 
should have a sign over the door 


SPONSE. 


narked that every psychology laboratory 
would read: STIMULUS and RE- 


rei 
which 
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of behavioral adaption and learning. In. view of the neglect of the 
problem of punishment alluded to by Dinsmoor and Stone, it may 
seem strange that this is the area chosen by the present writer as, so 
to sav, the cornerstone of what purports to be an extended and system- 
atic treatise on the psychology of learning; but it seems that, as in the 
story of the little girl who was told by the elves to “sweep under the 
rug." we too have found gold. However, there is still not a universal 
acceptance of the systematic position on which we propose, in this 
book, to build; and before proceeding with this undertaking it will be 
well to review and carefully appraise some of the misgivings and, in 
some instances at least, misunderstandings which thus exist. 


X. Objections and Criticisms Considered 


Central to the argument which has been presented in this chapter 
is the empirically derived assumption that there can be no adequate 
theory of behavior and behavior change which does not accept the 
reality and functional relevance of such subjective states as fear and 


(as later chapters will suggest) other emotions. In square opposition 


to this view is the behavioristic dictum that only observable, strictly 
"objective" events are proper data for a genuinely scientific psy chology 
—or behaviorologv. Gradually, over the years, much of the force 


has gone out of this stricture; but the position is still. occasionally 
defended. It is the purpose of the present section. to review some 
of the more energetic views thus expressed by latter-day behaviorists 
and other criticisms of the systematic premises on which later chapters 


of this book are founded. 

(2) Conditioned stimulus aversiveness versus fear, In his 1953 book. 
Science and human bebazior, B. V. Skinner states his position in this 
connection with stark simplicity. The chapter on "Emotion" begins 
with this sentence: "Ehe ‘emotions’ are excellent examples of the 
fictional causes to which we commonly attribute behavior (p. 160, 
italics added). Then the familiar, but highly debatable, assertion. by 
William James is approvingly quoted. to the effect “that we feel 
sorry because we ery, strike, or tremble because we are sorry, angry. 


or fearful, as the case may be" tp- 160). 
Rarely are theoretical issues (in psychology ) so beautifully explicit 


and "clean." The present writer, in company with many others, takes 
the position that the emotion of fear is causal, that a one-step (simple 
S—R) psychology will not work; that, as a minimum, two casual steps 
are necessary to account for behavior; and that fear; ia the case of 
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both active and passive avoidance behavior, is an essential inter- 


mediate "cause" or “variable.” 

v contrast, Skinner and those of like persu 
not at all the case and display great persistence in trying to suggest 
alternative wavs of dealing with the pertinent issues, One device com- 
in this connection is basically the same as w hat 
"reflex chaining” (1 lilgard & Marquis, 1940. 
to get around certain subtle 


asion hold that this is 


monly put forward 
carly behaviorists termed * 
pp. 210. 226). Here the attempt is 
| reinforcement by positing a purely “me- 
chanical” (unmotivated and unreinforced) linkage of stimulus-response 
1960, Chapter -). Or an attempt is made 
a quite transparent logical sub- 


problems of motivation anc 


(reflex) units (cf. Mowrer, 
di deal with the problem by means of 
terfuge: to deny the simple—and 
any avoidance-conditioning situation, 


instead that the stimulus 
and thus capable of motivating 


obvious? —assumption that, in 
! the CS or danger signal arouses 
fears and to affirm itself, in some rather 
Mexplicit. way, becomes “aversive” 
: reinforcing behavior. 


and (at its termination) 
cogent paper on punishment 


otherwise highly 


For example, in the 
find 


by Dinsmoor, already cited, we 
follows: j 


this strategy exemplified as 


amental to subsequent dis- 


stimulus is fund 
] will use the word in a 


Ince the concept of aversive 
cussion, T will begin by offering a definition. . « « *. n 
Strictly functional or behavioral sense, with no reference to its subjective 
Properties or to any assumed. drive which might be said to be aroused or 


reduced xeelie presentation or remov ively. of the stimulus. FT 
l lie critical observation is that tbe reduction or elimination of the stimulus 


Mereases the frequency or probability of the preceding bebavioral sequence 
thar is, thar it is reinforcing to the subject. 


this classification apparently in esc 
| water and certain intensities of light, sound, 


al, respect 


Vor the naive organism, includes such stimu- 
lating events as immersion ir 
temperature, and electric shock (pe 35): 

With the foregoing there need be no disagreement. The author 
of Ww others would. refer 


bolic”) stimuli having 
The difficulty arises in the 


is. heré : : o as extrinsic (as Op- 
here speaking of what 9 ^d l i 
E: inherent. (as op- 


Posed to intrinsic or “men 


posed to acquired) drive propertcs. ^ Ru 
Next step, when Dinsmoor tries to. explain “how neutral stimuli be- 
Come aversive.” Tle says: 

to be reliably established: that a neutral stimulus 


ue Main fact seems Oo) 
thins s presented just prie! £m 
i e Ey aversive stimulus, like à 
f “ight and becomes what we may 
aversive stimulus (p. 36)- 


or overlapping v ith the administration of a 
shock, acquires an av ersive property in its 
call a conditioned or secondary 
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Such an assumption works well enough, up to a point, as Dinsmoor's 
paper demonstrates; the trouble, as in all behaviorism, is that. the 
assumption is so disingenuous and, ultimately, nonfunctional. For 
some purposes it makes little. difference whether one assumes that a 
danger signal elicits the "aversive" emotion of fear, which then acts 
to motivate and (through its reduction) to reward behavior, or in- 
stead assumes that the danger signal itself “becomes aversive.’ so 
that its appearance is directly motivating and its disappearance re- 
w arding. Operationally .the two positions, in many situations, come to 
much the same thing. There are, however, limitations inherent in 
the Skinner-type approach which inhibit the kind of theory develop- 
ment with which we are especially concerned in this book. And there 
is, in addition, an absurdity: in the position itself. Dinsmoor, with 
other members of the Skinner school of thought, savs that the formerly 
neutral stimulus, as a result. of its pairing with ^ primary aversive 
stimulus, like shock, acquires an aversive property ii its oxen right.” ^ 
Can this, literally and actually, be truc? If the expression, "in dts own 
right" is to be taken seriously, it presumably means that the stimulus 
itself changes; 2 


and if this were to happen, it would subsequently atfect 
other organisnis aversively, and not just the oreanism which had, more 
or less adventitiously, been present when the formerly neutral and the 
primary aversive stimuli were "paired." An organism which is not 
present. when, for example, a buzzer and electric. shock are simul- 
tancously. presented to another organism (or, for that matter, in any 
empty box) does wor subsequently find that buzzer to be aversive. 
"in its own right." The "aversiveness" is the property of the affected 
organism, not of the stimulus itself: and this fact is fully recognized 
in the hypothesis that the conditioned stimulus acts as if it had become 
“aversive” for the reason that the organism now reacts to it with 


an "aversive" emotion, fear! 


Assuming that students. of learning were themselves electronic 
robots, rather than members of the animal kingdom, they would still 
have to take the foregoing considerations into account; and if they 
operated at all logically, they would have to infer an intervening 
variable, within tbe subject, rather than a change in the stimulus: 
Being sentient creatures ourselves, we certainly know, first-hand. 


the phenomenon of fear; so it is not its postulation, but its systematic 
rejection, that is strange and unscientific. 
(b) [s shock avoidance rewarding? A psychologist who read an 


`Sidman & Boren (i9sza) say thar “The stimulus itself becomes aversive” (P- 
282). 
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tinal comment to the effect 


a draft of this chapter made a ma 
ilie i : : à ; i . 
at “Ramin has found evidence that, in lactive] avoidance learning, 
n but also shock avoidance itself, 


may not be just escape from fe 
that ds r¢infareine T i 1 i 
Mos reinforcine;" The experiment here referred to was reported in 
1086 m P wip y 3 1 1 F 

256 under the title, “The effects of termination of the and 


avoi 2 at "ge 1 ing," Kami i 
adance of the US on avoidance learning.” Kamin summarizes his 


UNperiment. thus: 


U l he effects of response termination of the CS and of avoidance of the 

5 On avoidance learning were studied with a 2x2 factorial design em- 
ploy ing 32 hooded rats as Ss in a shuttle-box. There were significant main 
“fees of both factors, with no interaction, The avoidance of the US and 
response termination of the CS both resulted. in more responding, and in 
The effect of avoidance of the US was tentatively 
attempt. was made to interpret the effect of 


shorter response latency. 
ri saat in S—R terms; no at w ve 
Ponse termination of the CS in cognitive terms (p. 423). 
Kamins results are here reproduced as Fig. 2-4. ‘The performance 
of the cight animals in the “normal” group. « herein a response (run- 
Ding to the other end of the shuttlebox) made to the danger signal 


(buzzer) had the conjoint effect 
Preventing the occurrence of shock, 
other three groups. Here the procedure may 
doulle “instrumental” one: the running response not o 
Mmediate escape from fear (through buzzer termination) but also 


of terminating the signal and of 
is clearly superior to that of the 
be said to have been a 
not only provided 


averted the shock (Mowrer & Lamoreaux, 1942). In CORTES the 
Classical” procedure, wherein response to the danger signal neither 
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terminated. the signal nor averted the shock. gave decidedly inferior 
results—consistent. with other. findings previously reported in this 
chapter. 

Thus far the results of the Kamin experiment tell us nothing new. 
zut what we now note is that the subjects in the other two (inter- 
mediate) groups apparently benefited about equally from cach of the 
two instrumental procedures, alone: shock avoidance and buzzer 
termination, with a slight advantage going to the shock-avoidance 
group (cf. an experiment by [raum & Horton, reported by Mowrer, 
19502, pp. 287 tT). What does this finding mean? 

As will be presently indicated, the Kamin experiment is somewhat 
confounded in design, or at least in interpretation, and its results 
cannot be taken at face value. But the study usefully serves to bring 
up an issue which has not vet been fully considered (cf. Section H1). 
When it was becoming apparent, some years ago, that "instrumental" 
training procedures give better. results than does the "classical" pro- 
cedure, it was sometimes objected that this just could not be the case, for 


it would involve teleology. i.c. causation from the future rather than 

from the past (or present )—w hich is contrary to the basic assumptions 

of science. However, by the. postulation of fear as an intervening 
ble the specter of teleology was etfectively banished. 

The finding that laboratory: animals more readily respond to a dan- 


ger signal if their response averts shock (or some other noxious 
stimulus) of which the signal is premonitory may w cll give the ap- 
pearance of their behaving as they do "in order to” avoid the shock. 
This, of course, would be rank teleology. But if we assume, instead, 
that as a result of paired presentation of signal and shock, there is an 
associative shifting of fear from the latter to the former, then we 
scc that, upon presentation of the signal, fear will be aroused and 
the subject will respond in a particular way because the signal pro- 
duces the fear and the response in question gets rid of both. (This 
resolution of the paradox is today widely accepted, even by those 
writers who insist upon speaking ‘of "conditioned. stimulus ave "son 
rather than fear; sec, c.g... Dinsmoor, 19545 Schoenfeld, 19505 Skin- 
ner, 19533; etc.) 

Now the Kamin experiment, in an ingenious but somewhat. mis- 
leading way. calls this analysis into question. It implies, as the reader 
of this chapter pointed out, that there is, after all, something about 
shock avoidance per se that is reinforcing, quite apart from the fear 
reduction which termination of the danger signal provides. In Kamin's 
words, “While a combination of CS termination and US avoidance 
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een ile oie responding, either ot these factors alone adds 

gia y to response strength (ps gees it valid, this is a serious 
ation and must be considered. 

ea Es : E prblem llv. about the results reported by Kamin 

erminate-CS” group. Here the procedure differed from 

in that the CS was ended by the 


a 


that of the “classical” group only 
face that performance is superior in 
group speaks in favor of the hypothesis that fear 
CS termination, is a reinforcing event al- 
will not be ay reinforcing 


ubjects r 
subject's response; and the 


The “terminate-( 
reduction, mediated by 
though, obviously, CS termination here 
a shock (“normal” procedure). 
hes ^s real issue hinges upon the i iven to the results 

amin's "avoid-US" group. Here, ostensibly. the CS is not re- 
only the avoidability of the US can account 
ince of this group. W hile not so good as 
is still reliably better than that of the 
intimation that the danger 


as it |] e * 
tis where ir is never followed by 
interpretation m 


Sponse-terminable, and 
or the face thar performane 
we of the “normal” group, 
fallacy. lies in the 
; bviously 
“With an avoid-US subject, a 
that trial but had no effect on 
after its onset" (p. 4215 


“classical” group. Phe 

Signal, or ES was interminable. O this was not the case, for 
i. author specifically says that: 
i caused the US to be omitted for 


e CS, which continued. to act until 5 


italics added). 

Fhe difficulty: lies; of course. in the assertion that “a CR... had 
10 effect on the Ir is true that, in the “avoid-US" group, 
@ response was not uniformly followed by immediate CS termination 
but CS rermination did occur eventually, in 
five seconds; 50 that what is here involved, 
on of response and CS termina- 
As will be shown in Chapter 
laboratory. animals 


SCC. 


SAP fear meducman)s 
OR died soon—after only onc 
anifestly, is nor a complete dissociati 
reward. 
rats and other 
where the responsc- 


Uic $ ( 
D but rather just a delay of 


I on 3 : 
it is now well established. that 


of delayed rewa rd. 


for that matter, considerably: 
t so efficient. as it is with in- 
{seem that what Kamin has 
is reinforcing but merely 
tion), following response, 


n tuer under conditions $ 
] ard interval is five seconds (or 
)nger), although such Ie 
stantancous reward. ‘Therefore. 
Shown is nor thar US avoidance. 


th: ope oh 
UU delayed CS termination (and feat 
is immediate 


learning 
it woulo 
as such, 
reduc 
is less santorini chan CS termination, This is quite 
different frait purportedly proving thar shock avoidance, as such, 
'S reinforcing.” ! à 


Por a similar i 20 os Woadworch & Schlusberg: (i954). 
m OT a similar interpretation. see Waou fi " 
see Kamin (193727- 


For d 
Y a sequel to this study. 
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(c) Avoidance learning and tbe "contiguity principle.” “The ex- 
periment by Brogden, Lipman, & Culler (1938) has already been 
cited (p. 42) as providing onc of the first and best. demonstrations 
of the superiority of an instrumental training procedure in active 
“Avoidance 
‘Training and the Contiguity Principle” in which he criticized the in- 


avoidance learning, In 1948. Sheffield published a paper on 


terpretation commonly made of the Brogden-Lipman-Culler findings 
and advanced a different one. His first point is that too much has 
been made of the inability of the “contiguity principle" to account 
for results of this kind. As will be seen in Chapter 8, the term “con- 
tiguity principle,” as it is employed by Sheffield and others. who 
subscribe to à conception of learning put forward by F. R. Guthrie, 
has a somewhat clastic meaning. But eve of its meanings is the re- 
flexological notion that conditioning is simply a matter of a given 


item of overt behavior getting shifted, as a result of the contiguous 
presentation of a signal and an unconditioned stimulus, from Uncs 
to signal. Therefore, in the type of situation under. discussion, if 
electric shock causes a guinea pig in a revolvable cage to jump for- 
ward or run, then this response should get directly connected to any 
stimulus which immediately precedes the shock-run sequence, without 
the necessity of positing fear or any other sort of intervening variable. 
But, in the experiment in question, this principle runs into the paradoxi- 
cal fact that 100/¢ (classical) reinforcement of the conditioned stimu- 
lus leads to decidedly poorer performance on the part of the sub- 
jects than does a procedure in which shock follows signal only part 
of the time, namely, on just those occasions when the subject does 
not respond to the signal. 


Sheffield makes the observation that in the classical conditioning 
procedure, in the experiment cited, the shock often comes on, not ws hile 
the animal is at rest. but (because of the action of the signal) al- 
ready running and that, under these circumstances, the effect of the 
shock is sometimes that of causing the animal to stop. Hence, the 
contiguity principle would hold that on ensuing trials the signal wou d 
tend to clicit stopping, rather than running—w hich would then ex- 


plain why 100^; -shock-reinforced animals would not show a high 
incidence of running to the signal. This, of course, is a complete s 


legitimate argument; but the facts admit of another—and more broadly 
applicable—interpretation, When an animal has been put into motion 
by the danger signal and is then shocked, obviously the effect, as 
far as the running response is concerned, is one of punishment, so that 


a condition of conflict (cf. p- 44) ds thus generated: the danger signal 
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arouses fear which says "go"; but the response-correlated stimuli 


aroused by "going" say “stop.” with the highly erratic behavior indi- 
cated by the lower curve in Fig. 2-3. 

Hence we may conclude that Sheffield's first. point is apposite: 
that, taken alone, the results of the Brogden-Lipman-Culler experi- 
ment cai be explained by the simple contiguity (Pavlovian) principle. 
But in other situations, some of which have already been mentioned 
in this chapter (but see also Chapter 3), the simple contiguity principle 
is manifestly inadequate. The alternative interpretation, which in- 
volves the concept of fear as an intervening variable, scems to apply 
equally well in 4/ the situations cited. “The formulation. with the 


greater generality obviously has the advantage. 

Sheflield’s second. point is based upon a rather common miscon- 
ception, In referring to the Brogden-Lipman-Culler study and his 
own replication thereof, Sheffield says: 


The findings for avoidable shock offer no evidence for a strengthening 
etfect of avoidance | compare the Kamin study ] and follow the conventional 
result in Pavlovian experiments. Reinforcement by shock strengthened con- 


ditioned running; omission of shock led to extinction. The picture of al- 
2 voidance fits well Hull's 


ternate strengthening by shock and w cakening by c j 
(1929) theoretical analysis of conditioned defense reactions, and Schlos- 
berg's (1936) description of his rats’ behavior. The findings do not at all 
p : A i argui " EUR ce 
fit the alternative contention (Hilgard and Marquis, 1940; Mowrer and 
Lamoreaux, 1942; Schlosberg, 1937) that omission of shock strengthens the 
response rather than weakens it. For example, according to Hilgard and 
Marquis, , n Dee 
"Learning in this situation appears to be based in a real sense on the 
avoidance of the shock. . - - In instrumental avoidance training the new 
in the absence of any such [noxious] stimulus; 


ause of the absence of such a stimulus” (1940, 


response is strengthened 
indeed, it is strengthened bec 
pp. 58-39). 

"The implication is that a lea 
would be found during consecuti 
ings show the reverse to be true (pp. 1747 


irning curve rather than an extinction curve 
i of shock. The present find- 


ve avoldanc 


According to principles previously considered in this chapter, the 
resolution of this paradox is simple and straightforw ard. In a situa- 
tion of the kind under discussion, on a trial when the subject responds 
to the danger signal and averts shock, zwo things happen: the trial 
is indeed zh extinctive one for the fear which the danger signal pro- 
duces, since the danger signal is not "confirmed" by the shock; but 


—and this is the main pornt—sue | 
behavioral response 1s concerned, since its occur- 


signal, with attendant fear reduction. It 


h a trial definitely involves reinforce- 


ment as far as the 
rence turns off the danger 
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entirely true that, in an instrumental avoidance procedure, 
Met quii es response will eventually stop occurring; but this is 
lar A tes ing fear reaction to the daneer signal has extin- 
tea jid gi esi il Aas "habit" of responding lo. the feat has 
p d The “habit may still exist in full force; bur if the 
acquired. drive which c: 


Is the habit forth is not present, either. be- 
ause it has not been stimulated or because it has extinguished, the 
cause i S c 

habit will not be manifested—just as the habit o 


fo pressing a bar for 


food will not occur unless the subject is properly motivated, i.e. 
hunery.!! ^ 
(d) Is tbe present. systematization "premature"? 


Finally, we may 
consider a criticism from vet another quarter, 


Following an approach 


first used by Estes & Skinner (1941). Brady & Haunt 


stressed the inhibiting, or “suppressing.” etfect of a danger signal 


Urges). haye 


upon on-going behavior—such as bar pressing, by a hungry rat, as 
a means of obtaining food. Here clectric shock 


is used, not specifically 
as a punishment for bar pressit 


but merely as an agency for im- 
parting to a tonc, a "clicking noise.” or some other originally neutral 
stimulus the capacity to "disrupt or interfere with an organism's on- 
going behavior” (p. 313). Nor does 

secondary punishment, just when 

protracted presentation of the sig 
influence upon all overt behavior or, in short, the effect commonly 
termed “freezing.” Can this effect. be subsumed 
already considered or does it 
Brady & Hunt conclude that: 


the danger signal occur, as a 
the response occurs. Instead, the 
nal has a generalized inhibitors 


under principles 
go bevond or perhaps contradict them? 


Many attempts have been made to order 


a motivational construct me 
havior in such experimental 
tion of the experiments summarized in 
contrast to this contemporar 
constructs. 


these diversities in behavior as 
diating the establishment. of instrumental be- 
situations (Miller, rositb). The gen l orienta- 
this paper appear in rather clear 
y emphasis upon acquired drives and similar 
cal approach may clearly be seen to re- 
at such monolithic ordering, prematurely embraced, 
€ important differences as well as significant similari- 
among behaviors which have in common some crucial 
at we cony entionally regard as emotion (p. 322). 


chapter what Brady & Hunt 
takes the form of the 


This more empir 
ect some concern th 
might serve to obscur 
ties and relationships 
dependence on wh 


In this 


refer to as “such monolithic 
ordering assumption that active and passive 
i | ashley (1950) has made 
this affective clement is 
extinction of the conditi 


but a waning of this aff. 


precisely 


the same point in 
aroused, 


the conditioned reflex doc 
oned reflex is not 


these words: “Unless 
Snot occur, So-called 
ling of the specific association. 
(pe ap. 


a weaker 
setve reinforcement” 
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ayei learning depend upon (a) the conditioning of fear to, 
OER independent and response-dependent stimuli and upon 
! endeney of living organisms to develop, as a "habit," what- 
d form of behavior most effectively eliminates the fear thus aroused. 
n those instances where fear is conditioned to response-dependent 
stimuli (as in ordinary “punishment” ), the effect is one of specific 
response interference. or inhibition; and in those instances where 
fear is conditioned to an independent. stimulus, the tendency is for 
the organism to be activated. in whatever way will remove it from 
1. Brady & Hunt are talking 
different: the tendency for an in- 
which can be neither escaped from 


this particular. stimulus or vce 
about an effect which is clearly 
dependent, external danger signal 
nor eliminated to havea nonspecific inhibitory effect upon all behavior. 
Such a stimulus makes the subject “cautious.” vet highly “vigilant,” 
igh biological utility. 

X ny the reality of the phenomenon 
at the same time it in 


reneralized “freezing” 


Ma way that probably has h 

Here there is no wish or need to de 
about which Brady & Hunt are talking; but 
; rinciples. This g 
n instinctive device for ensuring care 
situation; whereas. the 


2H MY challenges the stated p 
reaction to fear appears to be a 
in a nonspecific danger 
relevant to those acquired 
adjustments to fear which have been here termed active and passive 

ie that there is nothing demon- 


a T ‘ 

nd circumspection 
Mechani * i i s 
lechanisms discussed in this chapter are 


avoidance learning. We thus mav assull 
£ these principles, as will be still 


about embracing 


Strably "premature? : : 
y—and from ensuing chapters. 


More : eee 
Ore apparent from the next sector 


mental Learning and 


XI. Further Analysis of Instru 
f the Concept of Fear 


the Increasing Acceptance o 
this volume, Dr. Janusz Revkowski, 
of the University of Warsaw, was a 
of Ilinois during the carly part of 


t of this volume and its sequel. 


s mentioned in the Preface of 
kd the Department of Psychology 
\ siting Scholar at the University l 
1959 and at that time read the manuscrip 
ive questions which Dr. vski 

eden, Lipman, & Culler shows 
ü g procedure 


jd S a 
d z CV KOWSK hase i 
\mong other percept Reyk ki raised was 


The ex 


periment of Bre 
e learning an ins 


trun 
than does the classical condition- 


vski asked, would happen if the 
ed in an 


the followin pay 
rental conditionir 


rf 
that in active avoidance 
results in far more efficient behavior 
Mg procedure, What now. Dr. Rey koy ked 
strumental and. classical s were similarly sni 
Sperimental situation involving reward, such as food, rather 


pr cedure: 
positive 
th: A Soya Pee Jes 

Sh scape from shock and fear of shoc k 
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` rime ` ject was in danger 
in the Brogden-l ee aue "ipao, ed fae 
only when a specific AEN vade 7) was present, And if the correct re- 
r for "discriminative stimulus ) v l : 
ed v Ree; when this stimulus was on, the subject was rewarded 
trema of averting the UnCS (shock), i.e., the response was "instru- 
iron imagine a situation in which a hungry animal can obtain a bit 
of food by pressing a bar when a cuc stimulus (S) Is present. And, fai 
sake of completeness, let it be added that if the animal presses the bar at 
other times, i.e., when the S" is absent (Skinner calls this the S2 period 
or condition), nothing is accomplished, just as nothing is accomplished 
in the usual avoidance experiment if a response occurs before the C5. 
or “SP, appears (but cf. Chapters 10 and 12). The question now is, 
what would constitute an instrumental as opposed to a classical train- 
ing procedure in the ty pe of situation just described? Ihe answer would 
seem to go as follows. If, in the presence of the cuc stimulus, the hungry 
animal is rewarded (with food) when it makes the 
and not rewarded when it does not make this res 
“instrumental.” That is to say, 
is contingent upon the 


prescribed response 
ponse, the procedure is 
the procedure is instrumental if the food 


occurrence of the response, which, of course, is 
the familiar, common-place procedure used 


"habits" in general.? Whereas, 
“unconditionally,” 


in the establishment of 
if the food follows the cue stimulus 
ie. without reference 
on the part of the subject, the procedu 
to the situation so extensivel 


€ to any overt, behavioral act 
re is "classical" and conforms 
y employed by Pavlov and his followers in 
the investigation of the conditioned salivary response, 

What, under the two conditions just. described, 
comes? The answer is clear: if rew 
when the animal makes the prescril 
of the cue stimulus, the response wil 
when the stimulus is pre 


would be the out- 
ard is forthcoming when and only 
red overt response in the presence 
l occur increasin 
sent. But if the rew 
willy-nilly, the overt response will not be 


gly when and only 
ard follows the cue stimulus 
learned. The animal will, of 
without qualification, may | 
must be distinguished 
free operants and discriminant op 
trol’). For example, a hungry 


RS This Statement; ^? somewhat misleading. Two 
kinds of “habits here, conveniently. labeled. by Skinner 
ants (habits which are under “stimulus con- 


: rat can learn a bar-pressing habit either (ay in a 
situation where it is free to press the bar and obtain food ar any time or (b) in a 


situation where bar pressing will produce food only if some special cue stimulus 
(SP) is "on." Hence the above statement that habits "in general" are instrumental 

appears only if response occurs. But the situation specifically. under 
that wherein the habit is, in addition, under stimulus control, rather 
than being a so-called free operant. 


ie. reward 
discussion is 
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course, learn to expect food following the appearance of the S" or C5; 
and this state of positive expectancy (or hope) will be indexed, among 
other ways, by salivation (as Pavlov’s experiments amply showed); but 
only upon the occurrence. of the cue 


since the food is contingent 
avior, no such 


stimulus and not upon the occurrence of any specific beh: 
behavior will ordinarily be learned (but cf. Skinner, 1948; also Chapter 
10). 

The situation as regards instrumental 
is thus parallel to w hat is known about instrumental and classical pro- 
cedures in active avoidance learning. As the Brogden-Lipman-Culler 
—3), avoidance learning. prospers under an 
positive reward learning; and avoid- 


and classical reward. learning 


experiment shows (Fig. 2 
just as docs 
a classical procedure, just as does 


es, however, autonomic (emo- 
: fear (as indexed, for example, 


instrumental procedure, 
ance learning languishes under 
positive reward learning. In. both ¢ 
tional) conditioning can and docs occu 
by the pyychogalvanic reflex) in the first instance, and hope (as indexed 
by salivation; see also Chapter 3) in the second instance. But as far as 
al) behavior is concerned, the difference in the per- 
by the. instrumental and classical training pro- 


overt (instrument 
formance produced 
cedures is striking and crucial. 


saving, presumably, that much the same picture also 


y (punishment) is concerned. 
i.c., when and only when it 


It goes without 
prevails as far as passive avoidance learni 
If an organism is punished “instrumentally, UM i i 
performs a given action, the ishment is effective, 1... the subject 
learns to inhibit this response (because the inhibition accomplishes some- 
thing), But if a "classical" procedure is used, ic. if the organism gets 
the noxious, painful stimulus (UnCS) regardless of whether it does Or 

he punishment soon becomes ineffective, 
at least as far as the control of behavior is concerned. Here, as the 
common saving goes, “You're damned if vou do and damned if you 

DU Il go ahead and do, if vou want to. 


pun 


does not perform a given act, t 


don't,” so vou might just as We 
Vhus the evidence is quite perv 
so-called 


asive that as far as overt action in 
general is concerned, a instrumental training procedure is 
decidedly preferable to the so-called classical procedure. Vhis proposi- 
tion is, in a sense, almost axiomatic and needs no special proof; for w hat 
it says, in effect, is simply that if behavior does not control what happens 
avior is futile and just as W ell not occur. On the 
yt seem to apply to emotional re- 
vorks very well. This is not the 
happens to such responses if an 
hem, too (cf. Chapter 11). But 


to an organism, the beh 
other hand, the same principle does nc 
actions; here the classical procedure v 
place to consider the 5 


question of wha 
cdure is used with t 


Instrumental proc 
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what present considerations do show is that for all ordinary behavioral 
acts, the classical training procedure is contradictory, anomalous, and 
oross|v inefficient, And as repeatedly indicated in this chapter, this 
EE of affairs can be satisfactorily explained only if one posits a two- 
step theory of behavior, with certain internal reactions commonly 


known as CWOtIONS intervening between objective stimulation and overt, 
S " P e 
behavioral action. 


Fear was the first emotion. whose functional significance became 


clearly apparent in this context, and it is noteworthy how widely its 


reality and svstematic importance are today accepted. There are, as al- 
ready noted, a few American psychologists who are still reluctant to 
concede that fear is a necessary concept in the study and understanding 
of avoidance behavior. But common sense is clearly against them; and it 
is surely not without significance, also, that at the Nencki Institute of 
Experimental Biology, Warsaw, Poland—which has a strong reflexo- 
logical (behavioral) tradition—there is today, under the leadership of 
D Jerzy Konorski, a marked tendency to accept the two-step con- 
ception of avoidance behavior. For example, in a recent paper by Fon- 
berg (1958), we read: 

The majority of the visceral and humoral reactions appearing [in the so- 
called experimental neuroses of dogs| (Gantt 1942, 1944, 1953, Gantt and 
Dykman 1952 Reese, Doss and Gantt 1953, Liddell et al. 1934, and others) 


are identical with the visceral and humoral reactions appearing to the de- 
fensive stimuli (Solomon and Wynne 195 


3. 1954. Dykman and Gantt 1954 
1956, Bersh et al. 1956, Wolpe 1952, 1954. and others). The appearance of 
these reactions in the defensive situation suggests that the above-mentioned 
visceral-vascular reactions form the basis of the anxiety state (Mowrer 1950, 
Solomon and Wynne 1954, Farber 1948). The fact that the pharmacologi al 
or surgical exclusion of the autonomie system (Wynne and Solomon 1955) 
weakens or even completely disperses the state of anxiety in the animal, 
handicaps the formation of defensive reactions, and causes their extinction, 
seems to favour this hypothesis (109-110). 


Thus the signals of the noxious stimuli—"imminent danger signals "— 
evoke a certain state in an organism, called the anxiety. state, which con- 
sists mainly in a number of visceral and humoral reactions. The neurotic 
state which is a state of “emergency,” heralding damage to the functioning 
of such an important organ as is the. central nervous system, would. also 


cause a state of anxiety similar to the state of expectancy for impending 
danger. 


The performance of an instrumental movement, a movement w hich al- 
ways before removed the threatening danger, would reduce the anxiety 
state (Mow rer 1950, Solomon and Wynne 1953, 1954 and others), thus 
providing a positive reinforcement. Therefore the appearance of the anxiety 
state would always produce a tendency to its reduction, that is, a tendency 
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to perfor ‘ 
db Ed si associated. with the reduction or abolition of this 
is wi oen herween the systematic position of the Nencki In- 
scotia SDS (sec also Beck. Doty. & Kooi, 1958) and the point of 

represented in this book will be further indicated in Chapter 8. 
e writers knowledge, the Nencki group is 
the concept of hope, it is believed that 
istent with their general theoretical 


Although, to the best of th 
Dot vet making systematic use of 
this concept is thoroughly consi 
position and that they will have n 


in huh Wood f ME 
i AS hich its functional significance 
u i be . ` E 
s book, It now appears that hope is no less essential than fear to a 


eenuinelv é E $ 
genuinely adequate behavior theory; and, as later chapters will show, 
bundant. However. before taking 


it will be necessary to consider 
ich follows. 


o fundamental objection to the way 
is elaborated in later chapters of 


its ^ SAN . E rd E f 
empirical justification 1s no less a 
up S n 

p this important strand of our story, 


Certaj ? i 
fun other developments, in the chapter wh 


XII ; 
* Conclusions and Forecasts 


atement, What our discussion in the 


"ecedi z ; song 

Preceding sections leads up to 15 the realization th 

Make two radically different hinds of reactions: overt, behavioral, in- 
iological” (autonomically 


ional or “phystolog ' 
function of the former Is 


event (avoid) undesirable happenings 
desirable ones. In other words, 
good: they must be "in- 
hand, the psychology" 
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Jut first, a sort of epitomtzing st T i 
at living organisms 


Strume > 
m ie responses ad emot 
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csigned to control ex- 
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working (WO 
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Mid deis sense that contractio! 
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xternal events or happen- 
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lowrer, 1960, Chap- 
the world and 
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our brains, between reality and mind. The emotions, so to speak. 
register, record, “accept” what is out there; but they don't do anything 
aol it, at least. not directly —that is the specialized task of our be- 
havior responses. But, as we have seen, the emotions play a vital role in 
instigating, guiding, and directing behavior, 

In other words, we have discovered that strict Behaviorism is not 
acceptable; that it is, in fact, grossly inadequate and has to be modified, 
at the very least, by admission of the concept of emotion. And when we 
thus acknowledge that there are two fundamentally different kinds of 
reactions, which serve two very different sets of functions or purposes, 
it is not in the least strange if we also seem to discover different laws of 
learning: conditioning, classical conditioning, that is—applicable to the 
emotions; and the Law of Effect (or “Instrumentality” )—applicable to 
behavior proper. Behavior has to be instrumental if it is to be selected 
and retained (otherwise it would be quite worthless, even harmful), 
whereas emotions are learned, it seems, on the basis of the “sheer con- 
tiguity" of stimuli, without any reference to what these reactions "ac- 
complish.” (Actually, we find that the emotions “accomplish” 


3 ereat 
deal—but not in the save way that behavior 


al reactions do.) 
Thus, to many students of the problem, it has looked as if a rwo- 


factor conception of learning is essential. This view will be explored, 
at some length, in the next chapter. But the reader should, at this point, 
have a word of w arning. In the end we are 

learning is reducible to conditioning; 
more chapters (four 


going to discover that all 
but it will be necessary for a few 
in fact) of the historical developments which we 
are here attempting to trace and understand to unfold before this 
hypothesis can be fully meaningful. 


Two-Factor Learning Theory: 
Versions One and Two 


As a result of the line of analysis pursued in Chapter 2, we now 


have a relatively simple and self-consistent explanation of both active 
and passive avoidance learning; but this may not seem to bring us 
noticeably nearer to a unified and comprehensive conception of learn- 
ing in general. When one looks away from the rather specific prob- 
lems with which the preceding chapter is concerned toward the 
ficld, the picture may, indeed, seem obscured 
things did not operate this way his- 
area of active and passive 


broader issues of the 
rather than clarified. However, 
Clarification of issues in the 


was followed by 
and it is with some of these that we shall 


torically. 
avoidance learning 
of far-reaching importance, 
now be particularly concerned. 


some notable developments, 


I. Two-Factor Theory, Original Version 


learning theory, did not come into 


common use until about 1950; but it was merely the name, and not 
its referrent, that was previously lacking. In the preceding chapters, 
we have seen the deep imprint made by the systematic views of 
both Pavlov and ‘Thorndike. Althou 
opposite in their formal properties. th 
exclusive. or, perhaps. complementary. Their 
former view. For example, in a paper 


The expression, two-factor 


gh these views were diametrically 
ere was an open question whether 


they were mutually 

originators tended toward the 

entitled “The Reply of a Phy siologist to l 
63 


svchologists," Pavlov (1932) 
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both been right—right about different but complementary aspects 
of the total adjustmental process. However, in 1943 another way 
of regarding the matter was advanced, with great persuasiveness. 
by Clark Iz Hull; and later developments with respect to two- 
factor learning theory cannot be appropriately reviewed until we 
examine Hull's alternative conception. 


Il. Hull’s Attempted Synthesis of the 
Views of Pavlov and Thorndike 


Attention has already been called to a stage in Thorndike's think- 
ing when he seriously considered the possibility that reward 
strengthens S—R connections but that punishment does not weaken 
them. This view was most fully explored in a book, The Fundamentals 
of Learning (1932b), which Hull claborately reviewed (1935) and 
which apparently made a deep impression ‘upon him. From it he 
seems to have conceived the idea that all learning might be dependent 
upon a single, unitary process of reinforcement, namely, that. pro- 
vided by drive reduction. (reward), and that such an assumption 
could be made to subsume under it both the Pavlovian and thc 
Vhorndikian forms of learning. With the strain. toward parsimony 
always strong in science, it was only to be expected that, since neither 
of the theories which Pavlov and ‘Thorndike had suggested had proved 
adequate to the needs of the situation, an attempt would be made 


to achieve a synthesis of the two which would replace 


i the loose 
sort of “fede 


S ion” implied by the conception of “two-factor” learn- 
ing theory sketched in the preceding section of this chapter. Hull 
(1943) went about this task in the following wav: 


iie sud e ied differences of opinion concerning the. relationship 
selective learning and conditioned-reflex learning, an explicit and 
somewhat detailed comparison of them as types will oM be P ends In 
order to facilitate such a comparison, | Fig. 3-1] has been ennseaicred to 
represent in considerable detail the dynamic factors here conceived to be 
involved in conditioned-reflex learning, in close sarallel ath tl : repre- 
sentation of the process of selective reinforcement ts z Sel ar 
VEL 


Varlier in the same chapter Hull had postulated, in keeping with 
the positive half of "T horndike's Law of Effect ME if : Gs se 
is made by an organism just before the RAGING Se der uae 
as that emploved in the example shown 
will become more. firmly 


of a drive such 
in Fig. 3-1, that response 
: connected with that drive, so that when 
the drive recurs, the response in question will tend to occur more 
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promptly and more vigorously. In other words, it was here assumed 


that living organisms are so constructed that responses which “solve 
(end drives) are “stamped in” and, in the future, will 


problems” 
assumed to be the essence of selec- 


be more likely to recur. This was 
tive learning or habit formation. 


Fig. 3-1. Hulls diagram (slightly 
modified) for showing how selec- 
tive learning and conditioning can 
be explained by a single set of as- 
sumptions. If a drive, such as that 
provided by electric shock, pro- 
duces (in a dog) a response such as 


leg flexion and i 


the drive goes off 


shortly after this response occurs, S. —4 
the connection tid arrow) be- Buzzer Fy ES 
tween drive and response will be Apparatus S x 
strengthened (selective learning). € 77» Ry (Lifting of foot) 
Likewise, it was reasoned, the rein- 
forcement produced by drive re- ^ 
Drive (shock) 


duction should also strengthen the 


connection (broken arrows) be- 
tween any other contiguous stimuli 
(SoS, “and the response in ques- 
tion. When the latter connection 
gh to cause Ra 


becomes strong enou 
to occur in advance 
"conditioning" is said to be mani- 


fested. 


of the shock, 


n. in addition, is to suggest how the same 


drive reduction, can also account for con- 


such as S and S. are present when the 
it was argued that the same reinfore- 
) should likewise strengthen the 


esponse R,. In other words, 


What Fig. 3-! does the 
sort of reinforcement, 1... 
ditioning. If certain. stimuli, 
D—R, connection is reinforced, 


ing event (at the termination of D 
5. and S. and the r 


the termination of drive 
and Ra, the same event ought also to 
other stimuli, simultaneously 
type of connection gets strong 
be said to occur: Eea Ry will 
vance of D. Ry is then said to 


connections between oe 


it was reasoned that if can strengthen the 
connection between the drive 
the connection between 
And when the latter 
tioned response? may 
or to S, alone, in ad 
r “anticipatory.” response. — 

fore, that in habit formation, c 
is formed (or at least. strength- 


strengthen 
present, and Ry. 
enough, a “condi 
occur to S. and S. 
be a “defensive,” o 

We may say, there 
dikian ) learning, the ¢ 


ər selective (T horn- 


onnection that 
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ed) is between some response and the stimulus drive“) whose 
D AA srovides the reward or reinforcement; whereas in con- 
enue Bitten learning) the connection that is formed is be- 
tween the same (unconditioned) response and some stimulus other 
than the one whose reduction provides the reinforcing state of atfairs. 
In an earlier work, the writer (19502. p. Sy) supr “sted that the te 
intrinsic and parasitic. might be used, respectiv ely, to indicate these 
two forms of reinforcement. The point of Hull's analysis was, of 
course, to explain both forms of learning with one and the same 
source of reinforement—hence the common reference to his theory 
as monistic. Of his theory Tull said: 


These last considerations suggest that the differences between the two 
forms of learning [trial-and-error and conditioning] are superficial in 
nature; i.c, that they do not involve the action of fundamentally different 
principles or laws, but only differences in the conditions under which the 
principle operates. . . . On one critical point both Cases are identica the 
reinforcing state of affairs in cach consists in the abolition of the shock 
injury or need, together with the associated decrement in the drive and 
drive receptor impulse, at once after the temporal conjunction of t 
ferent receptor discharge and the reaction. This is, 


we af- 
of course, all in exact 
conformity with the law of primary reinforcement formulated above 

Pavlov differs from the law of reinforcement by regarding 
element of the reinforcing state of affairs the occurrence of S in this case 
the onset of the shock.! On the other hand, the critical element in the re- 
inforeing state of affairs by our own hypothesis is the reduction in the 
drive receptor impulse which accompanies the reduction of the need, ic. 
reduction of the phy siological injury of the feet, caused by the termination 
of the shock. ` 


as the criti 


It is an easy matter to show the inadequacy of Pavlov's formulation as a 
general theory of learning. . . . It is not difficult to understand how Pavlov 
could have made such an error. His mistaken induction was presumably 
due in part to the exceedingly limited type of 


experiment which he em- 
ployed (pp. 78-79). 


This scheme of Hull's for explaining both response substitution and 


stimulus substitution in terms of the same principles can be perhaps 
most clearly. portrayed by the type of diagram reproduced in Fig. 
3-2. This is similar to the diagram in Vig. 3-1, except that no attempt 
is made to represent the experimental situation. Moreover, there is 
a further elaboration of the two kinds of learning supposedly in- 
! Although, at an carlier point, Hull 


indicates that the experimental situation 
under discussion is more like the 


ones typically employed by V. M. Bekhter 
(1913), he nevertheless felt thar it fitted into the 
of Pavlov (192 


general theoretical framework 
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n although instrumentally effective for deal- 
ing with the primary drive (lower line), in the beginning is slow 
to occur (sce broken arrow from drive to R,). But then, because it 
reinforced by drive reduction, it 
"moves. forward” and eventually becomes the first response made 
arrow from drive to R;). Here we have 
onses (not shown in diagram) which 
d to the drive are eventually dis- 


volved here. Response R 


is “correct and is consistently 


after drive onset (see solid 
response substitution in that resp 
are initially more strongly connecte 


another, originally w caker response tend- 


placed and superseded by 
ency. 


Fig. 3-2. Schematic representation 
of Hull's monistic C'one-factor) 


zeurea ; theory v Elere both Signal v n 
response substitution dower line) R, Ri 
and stimulus substitution (top line) 

were assumed to depend upon drive 1 
reduction, Drive induction, or on- Drive a 
set, was assumed to have no rein- 


forcing properties. 


learning or response substitution 


this. trial-and-error 
(according to Hull's 


becoming connected 
roken arrow from signal to Ry); and as 
R, again "moves forward" and 


But while 
is going on, R, ds also 
theory) to the signal (sec b 


this connection becomes strot À 
to the signal alone (see solid arrow from 


litioning, or stimulus substitution, in 
capable of being clicited only 
comes "associatively shifted" 
But Ri. 
instead 


wer, 


eventually starts occurring 
Here we have cone 


signal to Rj). 
was originally 


that a response which 
onditioned stimulus) be 


new, formerly i 
“instrumentally effective 
impinges upon the 


by the drive (unc 
(Vhorndike’s term) to a 
as a conditioned response. 
of merely terminating the 
organism, it now prevents 
the organism for 


neffective stimulus 


is still 
drive stimulus after it 
the occurrence. of this stimulus or other- 
: > sti sf. Spence, 19514, p. 
wise prepares the stimulus (cf. Spence, 19512, p 
262 1T.; Woodworth, 1958). 


Ill. Difficulties Inherent in Hull’s Theory 


4s theory, this is by no means a bad one in that, by means of onc 
and the same principle. 
for. But the theory involves a number 


some more so. Scveral of these cluster 


seemingly diverse phenomena are accounted 
of difficulties. some less serious, 


around the problem of punish- 
F 
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ment. In one way this is not surprising, for punishment was a problem 
with which Hull never really attempted to deal. The term “punish- 
ment" does not appear in the index of any of the three books in 
which he developed his system, and he seems to have been curiously 
unconcerned with, almost unaware of, it. 

Let us therefore begin by asking what might be expected to hap- 
pen, according to the schematic version of Hulls basic theoretical 
concept shown in Fig. 3-2, if the 3 ignal" which precedes the advent 
of drive consisted of stimuli which are response-correlated. rather 
than independent. Suppose, for example, that a hungry rat makes 
some food-getting, hunger-reducing response Ry, and that this re- 
sponse and its sensory consequences are followed by the onset of 


electric shock, which is terminated by a second instrumental response 
of some sort, Ri. We know, empirically, that, if the shock is relatively 
strong, Rin will be inhibited by two or three repetitions of this pro- 
cedure. How, within the framework of Hullian theory, can this out- 
come be explained? ; 

Hull has posited that learning occurs when and only when a drive 
is reduced. (cf. Chapter 4, Section H). The onset of electric shock, 
in the example cited, would have no reinforcement significance (either 
positively or negatively), and learning would occur only when the 
shock went off, thus providing positive reinforcement. ‘Therefore, 
one might expect that if, shortly before shock onset, the subject, 
under hunger motivation, had made some food-getting response, this 
stimulus-response connection would be strengthened, rather than 
weakened, by the advent of shock and shock reduction. Or it might 
be inferred, since the stimuli correlated with the occurrence of Rin 
shortly precede (and their traces—sec Chapter 1o—overlap) shock 
termination, that response Ris ought to get conditioned to these stimuli, 
so that whenever Rj, occurs it would cuc off Ry. 

Both of these inferred outcomes are, of course, contrary to fact and 
might seem to count heavily against the Hullian formula. We know 
that if a response such as Ry, is followed by strong electric shock, 
that response will be inhibited rather than strengthened and that its 
occurrence will vot cue off Ri. Hence it might appear that here are two 
straightforward deductions from Hullian theory which are empirically 
refuted. However, the system has unexpected resiliency: in this connec- 
tion. We must remember that in the hypothetical situation. described, 
hunger, though perhaps somewhat lessened by Rin is an on-going stimu- 
lus which overlaps shock and shock termination. Therefore, if the shock 
elicits Ri. this response, by the principles represented in Figs. 3-1 and 
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3-2, will become conditioned to the hunger drive; and since Ris is 
probably incompatible with Ra, the attachment of Ry to the hunger 
drive will make this drive Jess likely to elicit Rin. Q. E. D. 

Postulate 16, of Hulls system, 1943 version, specifically states: 


When the reaction potentials (Eg). of two or more incompatible reac- 
tions (R) occur in an organism at the same time, only the reaction whose 
Momentary effective reaction. potential. (sFr) is greatest will be evoked 
(Hull, 1943, p. 34427 

Hence, eiven the conditions stated, a “theory of punishment" may 
the Hullian formulation. Since this formula- 
designed to deal with the problem of punish- 
at it nevertheless appears to do so. 
that such an explanation of 


be said to be implicit in 
tion was nor specifically 
ment, it is much to its credit th 


However, further reflection will indicate 
limited, rather special circumstances. 


ary that there be some response made 
by the subject which consistently’ and promptly terminates the “punish- 
ment" (shock) stimulus. Suppose instead that this stimulus is of 
fixed duration. This would mean that there was no Ris ie., no response 
which would terminate the shock and therefore no response which 
would tend to ect conditioned to the hunger drive, in competition 
with Ry. Yet one would hardly doubt that, for these reasons, the 
shock would fail to have an inhibitory effect on Rim Results reported 
by Mowrer & Viek (1948) show that shock of fixed (uncontroll- 
able) duration is eve more effective as a punishment than is shock 
on is dependent upon the occurrence of some specific 
the explanation of punishment which follows, 
Hull's basic theoretical concept falls 
an entirely isfactory explanation of punish- 
il circumstances provided in the hy potheti- 
at Hull. himself, did not think too highly 
deal with the problem of punis 1- 
with Livingston, Rouse, & 


punishment is valid only under 
In order for it to work it is necess 


Whose terminati 
response, Therefore, 
somewhat incidentally, from 


short of providing 


ment even under the spe 
cal example. And it appears th 
of the capacity of his system to dei 
ment; for in an article published jointly Livin ou 
“True, Sham, and Esophageal Feeding as Reinforce- 


ated the view that there is only one kind 


Barker in 1951, 0n 
ments," Hull in effect repudi 
of reinforcement: 

An ineidenmail aliservatiün recorded by Bellows and Van Wagenen sug- 
sasesdoar ch ates that this deduction might be verified. One of their dogs 
[UTE scat ales esophageally fed) ceased to try to eat. “The temporal 
John D. Davis for calling attention to this postulate and 


. lam indebted to Dr. 
its implications in the present contest. 
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muscles became quite atrophic.” It may be significant in this connection 
that the. food surreptitiously seized by the dogs. usually clogged in the 
superior stenosed opening and caused retching. This clogging in the upper 
fistula presumably constituted a rly effective punishment in addition toa 
failure of normal nutritional reinforcement. This still further complicates 
the interpretation by suggesting the operation of negative reinforcement 
(punishment) as distinguished from experimental extinction (pp. 243-2445 
italics added ). 


The notion of negative reinforcement is foreign to the basic as- 
sumption in Hull's system that all reinforcement is positive (rewarding. 
drive-reducing) and can be articulated with his system only through 
the oblique device of relating it to the notion of reactive inhibition 
(or fatigue) w hich Hull employed to explain extinction. Therefore, 


1: 


A possible related observation made during extended sham-feedings is 
that our dog frequently showed “violent nervous shaking? (quoted. from 
the laboratory notes), presumably from the fatigue of eating, which evi- 
dently was considerable. It seems likely that this, possibly coupled with the 
enormous incidental secretion of saliva, may have been an important nega- 
tively reinforcing (punishment) factor in c using the cessation from sham 
eating which we have tentatively labeled “experiment extinction." On the 
other hand, the Mowrer-Miller hypothesis (Hull, 1943) holds the lj; is based 
on a concept closely related to fatigue, which is akin to punishment (p. 
244)" 


immediately following the passage just quoted, we r 


Then there is vet another palpable defect in Hull's attempt to in- 
tegrate the facts of conditioning and trial-and-error learning. As 
will be apparent from Fig. 3-2. his scheme will, in any case, work 
only where the response which becomes connected to a danger signal 
is the same as the response made to the stimulus thus signalized. It 
is well known that the response which becomes anticipatory, in a 
conditioning situation, may be radically different from the so-called 
“consummatory” or “unconditioned” response. For example, if a 
? Earlier, in 1943. Hull had written somewhat equivocally on this score: While 


acknowledging the possibility that “there may be more than one mechanism of 
reinforcement,” Hull took the position that the evidence that drive onset can 


provide reinforcement (of a negative kind) was meager and ambiguous, "I hus 
the only critical evidence now available seems to favor the reduction or termina- 
tion hypothesis” (p. 82). “We shall proceed on the positive assumption that the 
termination of the need (or of its closely correlated receptor response) is a 
primary reinforcing factor, this hardly seems open to doubt, Fven if the onset 
of the need, or of the correlated receptor response, proves to have genuine rein- 
forcing capacity, the dynamics of behavior are such that it would nor have much 
adaptive value" (p. 83). The latter assertion suggests, as do other considerations, 
that Hull never fully assimilated the implications of this problem into his system. 
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man discovers that his house is on fire, probably the most adaptive 
make is to call the local fire department. But 
man, after the fire is out (or even before it 
starts), becomes afraid lest his house catch. fire. What would we vow 
expect him to do? Taking the Hullian formula quite literally, we 
would expect this man to call the fire department. What we would 
a person would get in touch 
nse considerably different from 


response for him to 
suppose that this same 


actually expect instead is that such 
with a good insurance agent—a respo 
the one appropriate to the fire itself. 

Laboratory instances in W hich marked 
and conditioned | response has been noted, rather 
t least one study has been published 


dissimilarity between. the 


unconditioned 


incidentally, are numerous; but à 
in which a successful effort was made to produce this effect deliberately 
(Mowrer & Lamoreaux, 1946). Let us imagine a rat in the type of 
shuttle box previously described. (Chapter 2). After the usual period 
of free exploration, the buzzer five seconds, followed 
by shock, which the animal 


side of the box. Soon the 
c shock comes on; and a few trials 


later it starts making "avoidance" runs, to the buzzer alone. But, al- 
though running has been defined as the solution to the shock problem, 
it is ‘not, in this situation, likewise the solution to the fear problem, 
created by the buzzer. The correct response, or solution, to the 
latter problem is now a response very different from running, namely 
a leap into the air. Consequently. when the animal runs in response 
to the buzzer-produced fear, this behavior does not terminate the 
buzzer or circumvent the shoek; the buzzer continucs for the full 
comes just the same, and the rat again has to 


is sounded for 
can readily terminate. by running to 
the opposite * rat scampers promptly to 
the other end of the box when th 


five seconds, the shock 
run to terminate the shock. 
The rat now becomes agitated 


second period when the buzzer Is on, 
"random" responses. Eventually, 


and, particularly during the five- 
just prior to the shock, will 
of more or less 
while the buzzer is sounding. When this 
ed off, and the shock is not 
reduction, and occurs 


engage in quite a range 
the rat is likely to jump 
happens the buzzer is immediately turned | 
presented. Jumping is thus reinforced, by fear 
with mounting frequency 

It is naturally easier for 
1 ase Which 


the response W hich permits escape from f 
ate-ne amine, But widbos little mere time and effort, the rat can 
response as a means of escaping from shock 


on ensuing trials. 

an animal to master a learning 
permits escape from the shock and 
- (and avoidance of shock) 


situation 


in which the respi 


also learn to make one 
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itself and another, very different one as a means ob escaping bcd 
me fear of shock (sec also Keehn, 1959). Hulls system is unable to ac 
the fes st s re 

eco A pee em Hull's attempt to make all learning 
Tror ete what happens. when a drive gabs id ioo 
see that, although it can plausibly account for oth aug d 
eee and certain forms of conditioning, there is inherent rks 
theory mast of the same weaknesses that were present in I MA qn 
and Pavlov's original formulations. And itis in addition, porteur 
inadequate as regards the phenomenon of punishment. hater rd 
8) we shall have occasion to examine Hull's work more minutely, 
but the present sketch will suffice for immediate purposes. 


IV. Pavlov’s Attempted Derivation of 
Trial-and-Error Learning 


Hull’s attempt to "integrate" the learning theories of 
and Pavlov, which has been discussed in the two 
attracted much attention 
general reaction to it was f. 
By contrast, an 


Thorndike 
preceding sections, 
and, despite its palpable difficulties, the 
avorable (for at least a considerable time). 
attempt which Pavlov made, in 
trial-and-error learning in terms of con 
almost no attention, at least in 
which will be traced in later 
while in certain important w 


1932, to account for 
ditioning principles has received 
America; vet, in light of developments 
chapters, this attack upon the problem, 


avs incomplete and even obscure, had much 
to recommend it. Because this endeavor by Pavlov to deal with trial- 
and-error learning is so little known ause it comes so close to 
the systematic position which gradually evolves in this book. the per- 
tinent passage from the 1932 paper will be quoted in full. It reads as 
follows: 


and be 


When we were working out a method of feeding an animal from a dis- 
tance at the time of the experiment | just discussed |, 
different methods. This among others: In front of th 
an empty pan, to whic " 


[ h a metal tube led down fi 
which held the dric 


l d meat-powder and usu 
for our animals at the tir 


tainer and the tube was 


we tried out many 
e dog there was always 
'om a container above, 


ally served to provide the food 
ne of the experime 


nt. At the junction of the con- 
t a valve, which was opened at the proper moment 
by means of air transmission, so that a portion 


of the powder dropped 
down the tube and came out into the pan where it was eaten by the animal. 


The valve was not in good working order and if the pipe were shaken 


some of the powder from the container would drop into the pan. The dog 
quickly learned to make use of thi | 0f his 


? : own accord shaking out the 
powder. And a shaking of the pipe took place almost continuously, when 
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the dog was eating the portion of food which was given it and in doing 
so knocked up against the pipe. Of course this is exactly what takes place 
in training a dog to give one bis par In our laboratory work, the condi- 
tions of life have in general done the teaching, but here, man forms part of 
the conditions. In the latter case the words “paw,” “give,” ete., the skin stim- 
ulation from the contact in lifting the paw, the kinesthetic stimulation ac- 


companyiug the lifting of the paw, and finally the visual stimulation from 


the trainer, were accompanied by food, ie, were bound to the uncondi- 
It is absolutely the same in the instance cited: the 


tioned stimulus for food. 
noise of the shaking pipe. the skin stimulation from contact with the pipe. 


kinesthetic stimulation in jostling against the pipe, and finally the sight of 
the pipe—all these became similarly connected with the act of eating, w ith 
excitation of the feeding center. This of course occurred through the prin- 
ciple of simultaneous association, presenting thereby a conditioned reflex. 
And here, moreover, two additional distinctly physiologic | facts appear: 
In the first place, the definite kinesthetic stimulation in this case is probably 
linked up by a conditioned setting (in the lower parts of the central nerv- 
ous svstem by an unconditioned setting) with the execution of those move- 
ments or the activity which produced it.—this kinesthetic stimulation. And 


second, when two centers in the nervous system are connected or joined, 
set in motion and pass from one to the other in both 


the absolute law of one-way conduction of nervous 
then in the case cited one must 
e direction between these 
lditional neurone connect- 
a stimulus undoubtedly 


nervous impulses are 
directions. If we accept 
impulses in all points of the 1 1 
assume an additional connection in the opposit 


centers, i.c, one must grant the existence of an ac 
ing them. When food is given on raising the paw, 4 
runs from the kinesthetic center to the feeding center. But when the con- 
nection is established, and the dog, under the urge for food, gives. his paw 
himself, obviously stimulation runs in the opposite direction. l can interpret 
this fict in no other way (Pavlov, 1932. PP: 123-124; italics added). 
Despite the questionable involved, this quotation 
contains a number of noteworthy suggestions. Fi st and foremost, it 
makes explicit reference to a phenomenon which Thorndike (and 
Hull) largely ignored: the proprioceptive and other forms of stimula- 
tion which are normally asith the occurrence of the learned 
en as habits. In Chapter 2 we have seen how crucial is 
-correlated stimuli in punishment, and, as later chap- 
apparent that response-dependent stimulation 
role in “positive” learning. Pavlov clearly 
but many other events had to transpire 
cciated, a quarter of a century later. 
lly significant paper by Pavlov, 


nervous s "stem, 


ncurologizing 


associated 


responses knou 
the role of response 
ters will show, it is now 
plays a no less important 
adumbrated this perception. 
before its full force could be appr 


noted this historica 
of history and follow them as thes 


second version of two-factor learn- 


Therefore, having 
we again pick up the threads 
lead to what may be termed the 


Ing theory. 
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v. Two-Factor Theory, Version Two 


Those writers, cited in Section I of this chapter, who first cia Hed 
the v«wo-faetor position werc relatively uncritical of die £i m 
ceptions that had been set forth by Thorndike and | avlov. The 
T of response substitution and of stimulus substitution were, 
so to sav, laid side by side and used, as the occasion demanded, to 


in” : > type arning, now the other. However, as 
"explain" now one type of le g 


noted in Chapters ı and 2, there were manifest W caknesses in baih 
types of theory, even within their own spheres of special FER Pul. 
and it gradually. became apparent that, even when combined, they 
were not entirely satisfactory and required further modification, 
The crux of the difficulty, as it emerged during the fourth and fifth 
decades of the century, was that neither Pavlov nor Thorndike took 
the phenomenon of fear into sufficient account. In their attempts. to 
be thoroughly. "objective," they had ignored a vital aspect of behavior 
dynamics, and their theories suffered accordingly. Therefore, the 
present writer, in 1947, in an article, “On the Dual Nature of L 
ing—A Reinterpretation of "Conditioning" and ‘P 
tempted to show, on the basis of evidence and 
advanced by numerous other Investig 
two theories would have to be 


n- 
roblem Solving,” at- 
arguments that had been 


ators (see p. 6s), just how these 
altered in order to be 


truly compatible 
and effectively complementary. 


No attempt will be made here to 


give the details of this earlier 
analysis, but the main results the 


reof can be summarized as follows: 

1. It is unrealistic and inaccurate (with 
tions) to speak of overt behavior 
conditioning, Oniy emotions 


being attached to new stimuli 
then 


a few unimportant excep- 
as being learned on the basis of direct 
(and especially fear) are capable of 
by the conditioning principle. They 
Operate as "intervening variables.” 4 

Thorndike’s analysis of habit formation (as opposed to habit 


elimination or punishment) was essentially sound but needs to be ex- 
tended in the following manner. 


€ ‘ Thorndike limited his researches to 
situations wherein the subject was Motivated by hunger, thirst, and 
similar primary, or drives. It is now clear that tri; 


“metabolic,” 


-and- 
"The exceptions referred to 
reflexes, such as the eveblink 
conditionable; bur they “are 
which constitute behavior i 
especially. concerned, 


are the so-called. sho 
and the knee jerk. 


rt-latency (skeletal-niusele) 


These appear to be directly 
certainly nor representative or typi 


n general And it is with the 


al of the responses 
latter that we are here 
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error learning mav also occur, with equal felicity, when the or- 


ganism is motivated by an acquired, secondary drive, such as fear. 

"hus redefined, conditioning and trial and error constitute. two 
categorically different forms of learning, which may be termed sign 
learning and solution learning, respectively. They can be further dis- 
ünguished. in these ways: 

1. Solution learning occurs when a drive is reduced, a problem 
solved; whereas sien learning occurs when a formerly. neutral stimulus 
accompanies drive induction, or onset. 

2. Solution learning is mediated by the central nervous system, sign 


learning, by the autonomic nervous system. 
i. Solution learning involves. the skeletal muscles; sign learning, 


glands and smooth muscles. 
4. Solution. learning involves voluntary resp 
learning, involuntary responses (emotions). 


yonses (behavior); sign 


are complicated and can be understood 
forms of learning into account, Active 
Here, what occurs first is 


Most learning situations 
only by taking both of these 


avoidance learning is a case in point. 
becomes conditioned to a formerly 


sign learning, whereby fear ’ 
4 with the organism now motivated 


neutral stimulus or signal; then, 
by this secondary drive, trial-and-error behavior occurs and a be- 
havioral solution to the fear problem is found. Thus. where Behaviorism 
restricted itself to the simple, one-step S—R formula, we are here con- 
fronted by the necessity. of postulating, minimally, a two-step, two- 
Stage formula: [Om s—R. where S is the danger signal, r the response 
of fear which is conditioned to it, and where s is the fear, experienced 
as a drive, which elicits (after learning) response, R. Thus, what is 
S. namely r (as an internal response), be- 
comes the immediate (internal) stimulus for R. Miller & Dollard (1941) 
have usefully characterized fear as a responsc-produced drive. Fear 
occurs first as a response, but it is a disagreeable, “painful” response 
ational properties. 

re conditioning and pure solution learning 
. alone; the second two-factor posi- 


clicited as the response to 


and so possesses motiv 

This is not to say that pu 
may not, in some instances, 
tion merely held that both 
instances of behavior change and that here they cannot be laid side 


by side but must be placed end to end.? 


occur. 
are necessary to explain certain common 


3 experimental evidence pointing to the contrary view, that emotional and 
anger signal 
), but see also Gantt (1949. pp. 49-30). 


behaviorial reactions to a d are learned and occur simultaneously, sec 


Dykman, Gantt, and Whitehorn (1956 
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= ces xti f learni E ummarized as fol- 
5 "uu 1 of learning was s 

‘his tw rocess conceptior 

This two pro 


lows: 


š that there are many and highly diverse sources 
Inganimapi hens Werer heory of learning which is here under con- 
mune agr tae M nue Torrar ed] a delimitation of the term "condition- 
Me aer E deu D aire s Extensión of the traditional concept of 
TE. EE € The Gime "conditioning" has commonly been used, er- 
E ooi d ue seems, to denote the process whereby a living organism 
pi epe » o ny response, skeletal or visceral, immediate or delay ed, to 
SONR a NN has aal value.” As we have seen, this usage is too broad 
: ien deni purposes. It now seems preferable to apply the terim 
ell ede to that and only that type of learning whereby dece 
(visceral and vascular) responses are acquired. By contrast, rewarc ‘iy i. 
has been previously conceived as applving mainly in those S UA : 
which the motive, or "problem," is an unlearned biological drive, i 35 
hunger, thirst, pain, etc. It is now clear that reward learning must be hae 
panded to include those situations in which the motiv e, or "problem," is 2 
learned drive, i.e an emotion such as fear or an appetite. Many responses 
involving the skeletal musculature, which have previously been. termed 
"conditioned responses," are, in the present conceptual scheme, not pas 
tioned responses at all. Only those responses which involve visceral anc 


vascular tissue and which are experienced subjectively as emotion are as- 
sumed to be conditioned responses. If an emotion, 
causes the skeletal musculature to be 
in secondary drive reduction, then the overt response thus acquired is here 
conceived as an instance of reward le: 


arning, not conditioning (Mowrer, 
1947, p. 121). 


or secondary drive, 
activated and if such activity results 


VI. Further Elaboration and Implications 
of the Two-Factor Position 


By mid-century so much w 
which fear becomes att 
the ways in which it c 
behavior that system 
into account, The 


as known about the circumstances under 
ached to previously neutral stimuli and about 


an then function both to motivate and inhibit 


atic learning theory had to take this know ledge 
consequences, some of w hich h 
in preceding sections of this ch: 


ing. For one thing, 
and a cause of lea 
haviorism. 


ave been reviewed 
Were numerous and far-rcach- 
nent of fear, as both a result 


'd the death knell for primitive be- 
19th. C -entury 


apter, 

this acknowledger 
arning, sounde 
During most of the 
occupied with the phenomenon of conscious). 
method of iny estigation w 

may use the letter, O, 
of interest. for these e 
letter, c, loc: 


+ psy chologists were pre- 
TSS 


» and their principal 

"looking within." If we 
ganism, the chief. object 
an be represented by the 
it Behaviorism has changed 


as introspection or 
to represent the org: 
arlier researchers (G 
ated in the center of the O. Bu 
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all this, ushering in what Boring (1946) has called the era of “the 
LU bean O without c—in fact, even the O was com- 
monly omitted and behavior was schematized in terms of stimulation 
In short, O disappeared and S—R took its place. 
ar without rediscovering the organism 


empty organism,” 


and response only. 


But one could not talk about fe 
and, indeed, restoring to it some of the internal, mediating functions 


which had traditionally. been associated with the concept of conscious- 


ness. 

In 1953 the present author described. these developments as fol- 
lows: 

In writing a review of contemporary learning theory in the summer of 
1952 for the Journal of Educational Research, it occurred to me that I had 
previously thought of conditioning, or sig, learning, and of habit forma- 
tion, or solution learning, as essentially parallel processes whereas they were 
really sequential. The work of Miller (1948a), Brown and Jacobs (1949), 
and others had abundantly shown that if a formerly neutral stimulus, or 
sign, becomes capable, through conditioning, of eliciting a secondary drive, 
such as fear, the latter can then serve as the basis for trial-and-error, or in- 
strumental, learning. However, almost no one had previously laid these 
two forms of learning end-to-end, diagrammatically; and I discovered that 


toring to living organisms some of the internal 


When one did so one was res à eo 
operations which the behaviorists had so pre-emptorily dismissed. 


In conventional diagrams for conditioning and habit formation, it is usu- 
ally. indicated that the stimulation and response represented by various S's 
and R's can be either external to or W ithin the organism; but the “connec- 
tion” between them is ordinarily represented by a very thin line and not 
much more. Woodworth, in 1921, suggested that this might be carrying 
scientific abstraction a bit too far and ventured to insert an “O, for or- 
ganism, between stimulus and response, giving us the now familiar formula, 
S—O—R. McDougall (1938) made a related suggestion, and R. B. Cattell 
(1950) has proposed that the simple algebraic expression R= f(S), be 
refined at least to the extent of positing that R = f(S + O). The psycho- 
analysts had, of course, for many years been talking about all manner of 
interesting things that go on W ithin organisms; but academic psychologists, 
once in full revolt against the consciousness idea, were loath to rcacknow L- 
edge it or any equivalent. But many contemporary psychologists will feel 
comfortable with the notion of a two-step scheme of the kind show n [in 
Vig. 3-3] in contrast to and as a synthesis of the one-step conceptions of 
sign learning and solution learning taken separately. (Mower, 1953b, pp. 


164-165). 


and schematization employed in Fig. 3-3 
ke them consistent with those emploved 
cited to represent stimulus substitution 


‘The particular notation 
were adopted in order to mal 


in other figures in the paper A ; 
and response substitution, separately. But an example of how Fig. 3-3 
Might work out in practice will perhaps be necessary to make this 
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-ombined two-step type of theory fully intelligible. Imagine that a 
eg m the urgency of hunger (s) has learned to obtain. food by 
ien Tepbnss Re. Now lec us BHppoSe that Ma RUE. 
punished, by means of an electric shock Si) for ep ee ed Un WER 
S; produces the fear reaction r md ance repanse "n iip 
thetic and other stimuli wheney er dt occurs, these stimu i (Sa gete E 
ditioned to the fear response r. The result is that Ry will be inhibitec 
and Ra, R4, or some other response which will 
not provoke shock) will t 


rat has been 


also satisfy hunger (but 
in a simple instance. of 
conditioning. 
and essential role. 


ake its place. Here, 
trial-and-error learning, we see how 


+ Or stimulus sub- 
stitution, also plays an integral 


Fig. 3-3. Conditioning an 
trial-and-error learning 
put “end to end," thus 
providing a more com- 
plete and more realistic 
form of behavior theory. 


I 


clearer terms; 


ater this same type of relationship will 


has been said to 
tor theory the 


be schematized in other, 
indicate that, whereas in 
two forms of learning were 
in the second ve 
rdependent, And it will also now be 
as that the latter schematization, 50 tO say, revitalized 
and revived interest in internal, as well as external, stimu- 
lus-response events, 


bur enough 
the first version of two-fac 
thought of as parallel or independent, 
they were sequential and inte 
apparent why it w 
the organism 


rsion thereof 


VII. Empirical Support for the 


Two-Factor Position 


Reservations conce 
outlined in the 
lowers of Hull y 
learning, or 


rning the two-factor conception of learning 
preceding pages have come mainly from those fol- 
vho, with him, have ds really only one 
reinforcement, proc 1 
activated by or 


to be consiste 


held that there 


and that that process is exclusively 
rive reduction, For 
ake the somewhat 
on the basis of dr 
ld in 1956 n 
put the issue squarely 


Css 
dependent upon d 
Dt it must t 
even fear is learned 
a symposium he 


such a theory 
paradoxical position that 
ive reduction, or req ard. In 
"multi-process" learning theories, Miller 
when he said: 
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In order to stick to a consistent drive-reduction theory of reinforcement, 
one is forced to assume that the reinforcement of learning to fear a painful 
event is not the onset of the pain, but its reduction. This seems to be a 
direct. contradiction of all that is reasonable. . . - lL agree that this applica- 
ton of the principle of. drive-reduction goes against common sense; and 
I feel uncomfortable every time I am forced to make it. Once in a great 
While, however, when common sense is tested, it proves to be wrong; an 
utterly unreasonable assumption, like a ray of light being curved by gravity, 
a heavy one, turns out to be correct, That 
sting on conclusive experimental evidence 


ora light stone falling as fast a 
is the reason for stubbornly ins 
(Miller, 19512; p. B 


In the meantime, new support for the view that fear is learned 
on the basis of incremental rather than decremental reinforcement 
has been provided by a number of studies. Let us consider the follow- 
ing experiment, carried out by Mowrer & Aiken (1954). Preliminary 
Observations had shown that if hungry rats first learn. to press a 
Skinner-bar as a means of obtaining a pellet of food and if the act of 
pressing the bar then causes a tone stimulus—which has previously 
been paired with electric shock—to come on, the resulting inhibition 
(secondary punishment) of the bar pressing provides a very sensitive 


| 
UncS ————~ — 

Fig. 3-4. Schematic representation cs " 
f the five different arrangements 
of the five different arrange UnCS 
of conditioned stimulus (CS). and 
unconditioned stimulus (UnCS) 
employed in experiment by Mow- Us 
rer and Aiken (1954. p.e 28). In i 
Group V the conditioned stimulus UnCS 


(a blinking light) was not immedi- 
ately associated with the uncondi- 
tioned sti Jectrie shock) in 

imulus (electric cs ree 
ANV way, but was presented alone, m 
"Wo minutes after the UnCS had ms — I 


terminated, 


cs 


Sap ne V 
UnCS — —— 


Seconds 
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f > extent to which the tone has acquired. the pa cc 
ines ed hA -ordinelv, in the experiment proper. sev cra gr "| 
prd : ^: cd eo tone and shock, presented in the various EU 
D. dud eom RE in Fig. 3-4. If fear conditioning is y nine 
em CX niei i the Shek comes on, then the best learning 
eui ene to occur in the group in which the tone stimulus 
imitate pnts shock onset. On the other hand, if fear con 


? ‘he : ‘k goes off, 
litioning is dependent upon what occurs when the shock g 
. s i Sire SE 
then the best learning would be expected to occur 


in that group in 
which the tone was presented just in advance of Moe ds gk 
As the results shown in Fig. 3-5 indicate, the former ventions irod with 
latter predication was confirmed: A tone which had been pa ie 
shock onset was much more effective in inhibiting bar por aT 
food than was a tone which had bcen paired with shock termin: 


Mean responses 


Fig. 3-5. ( 


Aurves show ing 
in rats. The inhibition was 
electric shock in the Various ways indicated in 
with shock onset proved i 


inhibition of a hunger-motivated bar-pressing s eus 
produced by a Stimulus which had been associated Ait ; 
Fig. 3-4. The stimulus associate: 
most effective 


OE : EX 
as an inhibitor (secondary. punishine 
as revealed by the curve for Group I. 


TWO-FACTOR LEARNING THEORY 83 


In a related experiment, by Mowrer & Solomon (1954), an attempt 
Was made to get at the same issues in à somewhat different wav. It is 
well known (see Chapter 10) that if a stimulus is immediately followed 
more effective than if the re- 


bv. reinforcement, conditioning is 
Hull posited that the reinforce- 


inforcement is. delayed. Morcover, 
ment provided by drive reduction is more effective if the re- 
duction is abrupt ‘rather than gradual. Therefore, it was decided to 
have a tone stimulus immediately followed by shocks of different 
durations and different rates of termination, as shown in Vig. 3-6. If the 
is fear conditioning is concerned is the con- 


reinforcing event as far 
junction of a signal with shock ozset, one would expect roughly the 


same amount of learning in all four situations, since the relationship 
of the signal to shock onset is constant throughout. If, on the other 
ir conditioning is dependent on what happens when the shock 


one would expect best learning in procedure 1 zs less good 
n procedure 


hand, fe: 


terminates, 
conditioning in the other three procedures—less € good i 
drive reduction is temporally more remote from the signal, 


II because 
procedures Il and IV because drive reduction is 


and less good in 
gradual rather than sudden. 


cs it 
— i 
UnCS 
Fig. 3-6. Schematic indication of the cs m~ 
four different. forms of uncondi- ~ i 
tioned stimulus (shock) employed —§ UnC$— — — L 
in Mowrer-Solomon experiment 
Cosg p. 19). The conditioned stim- 
ulus was a blinking light of three sec- e i p 
onds duration, which was followed « m i 
in each of the groups of subjects UnC$ 
(rats). by a shock of the duration 
and form shown. 
cs n ert 
uncs i 


Seconds 


When the effects of fear conditioning were tested (using the pro- 
connection with the Aiken experiment). the 
3-> were obtained. They are exactly the reverse 


cedure described. in 


results shown in Fig. 
of what would be predicted on the basis of the drive-reduction theory 
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f fear conditioning: Instead of procedure | giving the greatest 
[2] * 2 


amount of fear conditioning (because of the greater proximity and 
cue more precipitous nature of drive reduction), it resulted in some- 
what less effective conditioning (reliable at the .10 level of confidence). 

The notion that fear conditioning is dependent upon the conjunc- 


tion of a signal and shock onset, rather than shock termination, leads 


Mean number of responses 


Fig. 3-7. Curves showing inhibition. of hunger-motivated. bar-pressing behavior 
in rats, produced by a stimulus which had bee associated with the onset of the 
various forms of shock indicated in Fig. 3-6. The amount of inhibition mani- 
fested in Groups I, HT, and IV is virtually identical; and the difference between 
these three groups and Group I although probably representing a real trend 
(sce text), is not highly significant statistically. 


us to expect equally effective conditioning in all four 
procedures. This prediction is well c n 
II, II. and IV are concerned; but, as 
was slightly (though probably: rcli 


experimental 
onfirmed as far as procedures 
we have just seen, procedure | 


ably) less effective than the other 
three procedures. Re-inspection of Fig. 


tion. Let us posit that in all four 
of fear was learned, 


3-6 suggests a possible explana- 
situations exactly the same amount 


because of the constancy of the signal-shock- 
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onset relationship. But then the signal is followed, fairly promptly, by 
shock termination. It therefore might be expected to come to signify, 
along with fear, also some degree of anticipated “relief or “hope.” In 
What situation would the anticipation of relief, or escape from shock, 
be strongest? Clearly in. procedure I, where drive reduction occurs 
relatively soon and also suddenly. To the extent that this opposite 
kind of conditioning occurred, it might be expected to counteract 
the fear aroused by the signal; and since the clement of "hope" (sce 
later chapters) would presumably be strongest in procedure I, the net 
effectiveness of the element of fear (presumably constant in all four 
procedures) would be least in procedure I. Such a deduction is con- 


376 


stant with the findings shown in Fig. 3 


VIII. More Evidence for the 
Two-Factor Position 


It has sometimes been suggested that shock onset may appear to 
be the specific reinforcing event for fear learning because shock on- 
set may be immediately followed by a subjective reduction in pain, 
even though the intensity of the shock stimulus remains the same. 
This possibility: has been described by Miller & Dollard as follows: 

Electrical recording of the responses of afferent nerves indicates that the 
" at first a strong burst of impulses 


Sudden onset of a new stimulus produces i À Wer i s 
from the sense organ which rapidly diminishes in strength till a plateau of 
; Si J28, pp- 67 and 116). This diminution is 


Stimulation is reached (Adrian, 1928. a à 
called adaptation. According to the principles w hich have been outlined, 
Such a reduction in strength of stimulation should, if marked enough (as 
might be the case following the sudden onset of a relatively strong stimu- 
US), act as a reward to any responses associated with it. C reful experi- 
ments may reveal that such a mechanism accounts for certain cases of learn- 
Ing which might superficially appear to be exceptions to a rigorous drive- 
Evi * 5 "t . E 
reduction theory of reward (1941. P- 35) 
In an ingenious experiment carried out recently by Davitz (1935), 
an attempt w as made to test the foregoing supposition, Using an electric 
À oradually but went off suddenly, Davitz 
gri A a x 
lus to one group of rats just as the 


shocle F 
hock that came on very 
Presented a blinking-light stimu ; dele 
auc en me hile another group of rats received the 
bias gradually came on, W hile anotl g I 

results shown in Fig. 3-7 can be seen in yer 
in the. place of a rat and ask: “Following 


lectric shock would I dread 


Vhe essential reasonableness of the 


Mother way, Let the reader put himself x 

the y arning signal, which of those four patterns of € 

OP fear Jesse” Vhere can be little doubt that the procedure-l shock form would 
+ T ere ci 


be the 
the one thus selected. 
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blinking-light just before the shock, at the height of its intensity, 
suddenly terminated, Since the shock came on so slow lv. it seems un- 
likely thar it produced any sort of Adrian “on-cffect.” vet it was 
found that the stimulus that had been associated with shock onset. 
gradual though it was, aroused considerably more fear (as measured 
by the capacity of the stimulus to inhibit free exploratory behavior in 
a standard test situation) than did the stimulus that had been as- 
sociated with sudden drive reduction (Fig. 3-8). On the basis of com- 
mon experience (and principles which will be considered later in this 
book), it would have been remarkable indeed if a stimulus which oc- 
curred just in advance of relief from the pain produced by the shock 
had become more fear-producing 


g than a stimulus which occurred 
as the shock was just starting to come on, 


80, = 
70) 

| ET Fig. 3-8. Results obtained from an 
60! “open-field” test. in an experiment 


e 
o 


| reported by Davitz Cross. p. 133)- 

ra In one group of subjects. (rats). a 
| stimulus was periodically. presented 
india | which had been associated with the 
| Beginning proup-—— onset of a gradually mounting clec- 
| tric shock (Beginning Group); in 
the ether group, the stimulus had 
been associated with the sudden 


w 
d 


Mean number of spaces traversed 
> 
o 


20 | termination of shock (End Group). 
z x The former was much more etfec- 
10 EN tive in inhibiting exploratory be- 
havior, thus suggesting greater fear 

0 conditioning. 


1 2 1 2 3 4 5 
Habituation Test 


Trials 


Some of Davitzs own observations concerning this experiment 
follow: 

The purpose of this study was to de! 
sented at the end of shock will a ‘quir 
presented at the beginning of shock, 

Hull (1943) has proposed a drive-reductio 
learning of fear. In his discussion of secondary motivation he suggests that 
stimuli acquire secondary-drive value as a result of association with the 
“evocation of a primary or secondary drive” and the “abrupt diminution” 


termine whether or not a CS pre- 
greater fear value than will a CS 


n hypothesis to account for the 
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of drive (p. 6). This suggests that the connection between cue and a 
fear response evoked initially by the onset of pain is aforced by a sub- 
sequent decrease of pain. “Abrupt diminution" of drive, or drive reduction, 
is the necessary condition of reinforcement; and, according to Hull's con- 
ception of the gradient of reinforcement, cues temporally closest to the 
occurrence of drive reduction are most strongly reinforced. If the begin- 
ning of shock increases drive and the end of shock decreases drive, a cue 
presented at the end of shock will be temporally closer to drive reduction 
than a cue presented at the beginning of shock. Therefore, the drive-reduc- 
tion hypothesis suggests that CS at the end of shock will acquire greater 
fear value than CS at the beginning of shock. 

In contrast to the drive-reduction hypothesis, Mowrer (1950a) has sug- 
a result of contiguity with drive 


gested thar a cue acquires fear value . l e 
Induction. Reinforcement in this ease is a function of increased pain rather 
than decreased pain. The fore, the drive-induction hypothesis. suggests 


that CS at the beginning of shock will acquire greater fear value than CS 


at the end of shock (p. 152). 


After briefly describing the Mowrer- \iken study previously men- 


tioned, Davitz says: 


However, Miller (19514) has suggested two other possible sources of 
drive reduction [other than the one provided when the shock was turned 
s a result of the animal's jumping 


otf]: (a) temporary escape from shock h pin; 
off the grid soon after the beginning of shock; (b) rapid adaptation of af- 


ferent nerves and consequent reduction of pain after the initial burst. of 
impulses at the beginning of shock | Adr i 1 effect]. Thus, in Mowrer and 
Aiken's experiment, CS at the beginning of shock could have been con- 
tiguous with one of these sources of drive reduction and the results of 
N accounted for by the driv c-reduction hypothesis. 

le present esperiment was designed. to minimize the effects of tempo- 
rary escape and adaptation in order to permit the assumption that the end 
narv source of drive reduction. To minimize the pos- 
ape from the grid, S, during the training period, 
inverted box which prevented it from jumping 
adaptation, a gradually increasing shock 


the experiment may be 


of shock was the pri 
sibility. of temporary ese 
Was placed under a small 


Vo offset the etfects of 1 
sec. Although an animal may adapt to one level of 


reasonable to assume that adaptation would be 


Of the grid. 
Was administered for 5 
Shock intensity, it seems n su A i 
minimized by a steadily increasing level of intensity (p. 152). 
Fhe results of this experiment have already been reported; they 


may seem quite conclusive. Nevertheless, Davitz suggests two pre- 


Cautions in interpreting them. 

riment, two possible sources of drive reduction, tem- 
minimized; therefore, a drive-reduction 
ondary sources of drive reduction was 


In the present expe 
porary escape and adaptation. we 


hypothesis based on these two se " 
contradicted by the data. However, another possible factor was noted dur- 
ing the course of the experiment. The Ss exhibited extremely vigorous and 


i á . activi ri sk xt of the s-sec. s tk. 
apparently disorganized activity during the last | f the s-sec. shock 
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1 ible that Ss in the End. group, preoccupied, in a sense, w ith the 

le posse B in of the shock, did not attend to presentation. of the CS. 
Increase B URS to fear CS when presented at the end of shock may 
ae Xütiodted. by Ss’ failure to perceive CS under the condition of 
DERE divo at the end of shock. : . 
SEE ae consideration is based on the particular behavioral measure 
FEX in this study. Freezing is directly: opposed. to the active type of escape 
or avoidance behaviors used in previous studies of fear. Although it has 
been assumed that the “fear” underlying inhibition of behavior is essentially 
the same as that underlying escape or avoidance behaviors, it would seem 
appropriate to investigate this assumption experimentally (p. 154). 


But in conclusion Davitz says: 


Despite the considerations mentioned above, the results of the present 
experiment, as well as the results of a previous experiment by Mowrer and 
Aiken (1954). seem to be most parsimoniously accounted for by the drive- 
induction hypothesis. This hypothesis suggests that a cue acquires fear 
value as a result of contiguity with increased drive and that reinforcement 
of the fear is a function of drive induction (pp. 154-155). 


Results similarly supporting the view that fear learning is contingent 
upon drive increment rather than drive decrement have been reported 
by Coppock & Chambers (1951) and also bv Bitterman, Reed, & 
Krauskopf (1952). However, others, notably Sullivan. (1950) and 
Zcaman & Wegner (1954), have reported | evidence which they 
interpret to the contrary, so it is too carly to say that the issue is 
absolutely settled. Nevertheless, it would now seem safe to proceed on 
the assumption that, in this particular instance, common sense has 
been right and that fear is learned on the basis of drive induction 
rather than drive reduction.’ Since it is at the same time agreed that 


* Since the above was written, the evidence on this score has become even clearer. 
Zeaman & Wegner, in the study cited, end with this statement: “The results are 
seen to favor the view that the mechanism of conditioning for this autonomic lly 
mediated response |heart rate] was that of drive reduction” (p. 354). However, 


Wegner & Zeaman (1958). treat the problem quite differently. They say, “N. E. 


Miller, for example, has stated that according to a strict drive-reduction theory of 
reinforcement *. . . other things equal, a signal followed by a brief noxious stimulus 
should acquire the capacity to clicit stronger fear than one followed bya prolonged 
noxious stimulus “ ( Miller, 19512, p. 375). And they conclude: “Despite the wide 


differences in Ss, procedures, and methods of inferring fear, the results of. the 


Mowrer and Solomon study are in agreement with our own findings. . . . The 


discrepaney between Miller's theoretical statement and our own data can not 
asily be attributed to lack of control of shock intensity" (p. 240). "lt is concluded 
that this proposition is cither incorrect or not testable under the conditions pro- 
vided” (p. 241). Miller himself has recently (1959). indicated that he is now “not 
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drive reduction is the reinforcing agent in other (ordinary trial- 
and-error) instances of learning, one is strongly tempted, on the basis 
of the evidence just reviewed, to adopt the two-factor position prce- 


viously described in this chapter. 

However, despite numerous attractions and advantages, the two- 
process, or two-factor, conception of learning—even in its two-step 
version—is not entirely satisfactory. In recent years, the present writer 
has felt and, on occasion, expressed growing discomfort. with the 
conception of "habit" tacitly accepted in the original version of two- 
factor theory (Mowrer, 19538, 1956). Moreover, the phenomenon of 
“secondary reinforcement” has come into increasing prominence in 
literature during the past decade or two, and even 
of two-factor learning will not at all explain in 
of the reported and well-authenticated facts. 


the experimental 
the second version 


this connection some 
It will therefore be our purpose, in the next chapter, to consider 
specifically the problem of secondary reinforcement, Doing so, we 
shall discover, leads (in Chapter 7) to a new interpretation of "habit" 


and to still further revision of the two-factor position. 


IX. The Chapter in Review 

we may sav that the first version of two-factor 
into existence because of the widespread con- 
of Effect nor the Pavlov- 
taken alone, could provide 


In retrospect, then, 
came 


learning theot 
viction that neither ‘Thorndike’s Law 
Bekhterey concept of reflex conditioning, 
aradigm for learning but that, taken together, they were 


a universal p DM EP eoa 
in Thorndike’s thinking, 


sufficient. Following an unexpected. turn 
Hull advanced the view that there is really only ove type of learning 
(reinforcement), namely that provided by drive reduction (reward); 
and on the basis of this premise he tried. to explain both trial-and- 
error learning and conditioning. However, Hull's theory, like the 
theories of Pavlov and Thorndike, could not adequately account for 


the drive-stimulus-reduction hypothesis". (p. 256); and, 


completely committed to Á j 
Up take this occasion to urge attempts to 


andor, he adds: 
test competing hypotheses, and time permitting, may 
(p. 257). And Spence (1956), another prominent 


with characteristic € 
formulate and rigorously j 
even join in that activity myself” Ts pt 
exponent of Hull's system, has now taken a completely different. position. con- 
cerning the whole reinforceme 
that the monistic thesis of Hull th 
conditions for learning has. in effect, collapsed. 


nt problem (sec also Chapter 8). So it may be said 
at drive reduction, and it alone, provides the 
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all l r acts of passive and active avoidance learnine, so the search for 
the fe $ ass à à : T 
more adequate and inclusive. conceptuali 


al ion continued. 
a 


‘The second version. of two-factor learning 


theory, like the first 


one, assumed that both conditioning and tr leand-error learning are 
real, and different, phenomena, but, unlike the first version, it made 
important changes in the way Pavlov and I horndike had identified 
and explained them. It was assumed that Thorndike (and Pull) had 
been right in positing that trial-and-error learning (problem solving) 
is contingent upon drive reduction; but thc conception of drive was 
extended to include fear, in addition to such primary drives as hunger 
and thirst. And it was further assumed that Pavlov (and Bekhterev ) 
had been right in holding that instances of learning occur, through 
sheer contiguity of stimulation, without drive reduction; but whereas 
the reflexologists had been interested only in objectively observable 
responses, the revised two-factor position stressed the emotion of fear 
the most important conditioned, or conditionable, response—w hich 
then motivates and when reduced reinforces, in a strictly Vhorndikian 


way, many of the overt, behavioral acts which Pavlov and others had 
tried to interpret as examples of simple, direct conditioning, 

Thus it was assumed that solution learning does indeed presuppose 
drive reduction but that sign learning, such as is involved in fear con- 
ditioning, depends upon the temporal conjunction of the sien with 
the onset of a noxious stimulus, such as clectric shock. Here one is 
dealing, not with drive reduction, but with drive induction, ic, with 
“punishment” rather than “reward.” Therefore, in the second. version 
of two-factor theory, not only were stimulus substitution and response 
substitution recognized as two different forms of learning; they were 
also assumed to involve two ditferent types of reinforcement, drive 
reduction and drive induction, re j 


ductior spectively." 
Vhis formulation was highly. efficient in accounting for active and 
passive. avoidance learning. But it was by no means entirely satis- 
factory. In accepting, essentially unmodified, 
habit as a stimulus-response connection or "bond," it was vulnerable 
to several telling criticisms that had been directed against this view 
(see Chapter 7; also Mowrer, 1960, Chapter 2); Altfeovts it was also 
evident that fear is not the only emotion that c 


that some emotions are acquired. in 


Thorndike’s notion. of 


an be conditioned and 
the contest. of drive reduction 
words, the assumption that there 
“Tf the reader would find it helpful to see the first 
factor theory represented graphically, 
Section I, Chapter 8, and the 


rather than drive induction. In other 


and second versions of two- 
he may consult the first four paragraphs of 
accompanying illustrations. 
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was a neat and specific correlation between conditioning and drive 
induction was palpably false; and, as if in recognition of these concep- 
tual difficulties, empirical research began to center more and more 
upon a phenomenon known as secondary reinforcement, which was 
to go far toward resolving them, So, once again, we follow history as 


well as logic and turn to this new topic. 


Two Conceptions 
of Secondary Reinforcement 


About 1940, the term "secondary. reinforcement" began to appear 
with increasing frequency in psychological literature; and. from the 
outset it had two, scemingly quite different meanings. The first. of 
these, we shall find, is merely an extension of 
primary reinforcement or "reward"; and the other interpretation, OF 
meaning, of secondary reinforcement is likewise an outgrowth of 
primary, or “first-order.” reinforcement. as Pavlov 
Since these two investigators thought of primary 


Thorndike's notion of 


conceived of 1t. 


reinforcement (i. 
rudimentary learning) 
al that the correlative 
reinforcement should also differ. 

These two conceptions will be separ 
and, in the chapter which follows 
they can both be accounted for 
assumptions. From this 


of the conditions essential for the occurrence of 
in such very different terms, it is only natur 
notions of secondary 


ately examined in this chapter; 
«à formula will be proposed w hereby 


in terms of a single, common set of 
analysis the possibility. then emerges of inte- 
grating and unifying Pavlovs and Thorndike's most basic and funda- 
mental hypotheses, i.e., of formulating à conception of learning with 
high generality and “power.” 


Secondary Reinforcement as Fear Reduction 


Since fear had, by 1940, been commonly recognized as an acquired 
or secondary drive, it was natural enough, as one group of writers held. 
92 
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to refer to the reinforcement of behavior through fear reduction as 
secondary reinforcement, secondary reward, or acquired reward, A 
with the title, “A Stimulus-Response 


paper which appeared in 1939. 
Analysis of Anxiety and Its Role asa Reinforcing Agent,” reads in part 


as follows: 


ated experimentally that, contrary to the 


John B. Watson (1926) demonstr: 
ated to and dependent 


Jamesian view, most human fears are specifically rel 
Upon individual experience. . + - 

Freud seems to have seen the problem in 
cordingly posited that a// anxiety (fear) re 


st in stimulus-response ] 
) stimulus (arising from external injury, or 


Whatever kind, or from severe organic need) impinges upon the organism 
and produces a more or less violent defense reaction. Furthermore, such a 
Srimulus-response sequence is usually preceded or accompanied by originally 
“indifferent” stimuli which, however, after one or more temporally contig- 
uous associations with the traur gin to be perceived as "dan- 
ger signals,” i.c, acquire the capacity to elicit an anxiety reaction. This latter 
reaction, which may or may not be grossly observable, has two outstanding 
Characteristics: (a ) it creates or. perhaps more accurately, consists of a state 
of heightened tension (or “attention” ) and a more or less specific readiness 
for (expectation of) the impending. traumatic stimulus, and (b), by virtue 


of the fact that such a state of tension is itself a form of d comfort, it adap- 
from the danger situation, thereby 


tively motivates the organism to escape l: 
y also probably decreasing the chance 


essening the intensity of the tension and y S i 
of encountering the traumatic stimulus. In short, fear is the conditioned form 


Of the pain reaction, which has the highly useful function of motivating and 
reinforcing behavior that tends to avoid or prevent the recurrence of the 
pain-producing (unconditioned ) stimulus (Mowrer, 1939. PP: 3547555 

1 effect, this: fear is acquired by con- 


ditionine and has the power not only to inhibit responses (if the un- 
has followed these responses) but also, if the 
q some external, environmentally 


ct to activity and to reinforce 


this licht from the outset and ac- 
actions are probably learned; his 
terminology, runs as follows. A 


hy pothesis, when re 
so-called "traumatic painful” 


natie stimulus, be 


What was being said here is, ir 


conditioned stimulus 
unconditioned stimulus has followed s 
Produced stimulus, to motivate the subje: 
es to eliminate the 
uence of Thorndike, there had been 
à tendency to think of a noxious, “punishing” stimulus as simply and 
er response it followed (Chapter 2); and, 
Pavlov and Bekhterev. there was also a 
i noxious stimulus as the "unconditioned stim- 
> response (such as leg flexion in the example 
Ced. from [Hull in the preceding chapter). « hich stimulus, if. paired 
incidental stimulus, would cause this response 
that stimulus. These two highly influential 


Whatever response serv “danger signal” and thus re- 
duce fear. Because of the great infl 


directly \ cakening whatev 
because of the work of 

fendeney to think of : 
ulus” for some reflexive 


Wi an 
ith a formerly neutral. 


fo: Has ; 
© become attached to 
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: MEE mE end, n: 
re represented diagrammatically in Figs. 4-71! and 4-2. 


-onceptions bons x z dne 3 
T ei “Thorndike, in his theory of punishment, was talking of 
x aching quite different from what Pavlov and Bekhterev were sug- 
some > ^ " "Ww x A E: Le b 
ting in their theory of defense conditioning, what is being sald here 
gesting ; - 


is different from both these theories; and this different way of think- 
is 


ing was prompted by the fact that neither lhorndike's theory nor the 
Fig. 4-1. Thorndilian concep- 
tion of how punishment oper- 
ated. If some drive, such as hun- 
ger, causes the subject to make 
some forbidden, “w rong" re- 
sponse and if this response ds 
followed by a painful stimulus 
such as electric shock, then the 


Hunger 


connection between the hunger 
drive and the “w Tong" response 
would, by virtue of the occur- 
rence of the shock (sce arrow), 
be weakened, 


Pavlov-Bekhterev. scheme was sufficient to the facts, as past. chapters 
have indicated. The present. position is that whenever 


à stimulus—be 
it response-produced (Fig. 4-1) or environmentally 


produced (Fig. 


Fig. 4-2. The Pavlov-Bekhterey vons 
ception of defensc-reflex conditioning. 
Ifan unconditioned stimulus, like elec- 


N =a SI trie shock, elicits some response R 

Buzan R SR {solid arrow), and if some incidental 

zt stimulus, such as a buzzer, is present 

Shock when R occurs, then the buzzer- 
Ci 


Fesponse connection (broken. arrow) 
will be strengthened and the response 
Will tend to occur to the buzzer alone 
(solid arrow), 

"Actually, since time is moving from left to 
should not be represented as weakening the Hur 
"bond? by “acting backwards” in time 
effect of the shoc 


right in this diagram, the shock 
iger—Wrong Response habit or 


fas suggested hy the arrow ). The true 
or punishment, must clearly be 


just after the punishment occurs, which etfect 
until the nest time rhe punished response st 


upon the "bond? just as or 
cannot, of course, be obse i 


Mts. T6 oceur. Xs already noted (Chap- 
ter 2), Thorndike, in speaking of his Law of I tfeci 


(and rewards) as “acting back” upon S 
arrow; bur this practice 
rather than 


often referred to punishments 
R bonds in the 


Manner suggested by the 
almost certainly represent 


ed mere carelessness of speech, 
a serious intention or studied conviction, 
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— precedes marked drive increment, fear becomes conditioned to 
that stimulus and can then act cither to produce response inhibition, 
if the signal is response-produced, or active avoidance behavior, if the 
signal is independent, i.c. environmentally produced. In both instances, 


doing Whatever will eliminate. the stimulus or stimulus constellation 
Which is producing the fear will. be rewarding and will reinforce 
Whatever activity (or inactivity) is involved. In short, by positing 
fear conditioning—which Vhorndike, Pavlov, Bekhterev, and Hull all 
eschewed—one can account equally well. and quite. parsimoniously, 
for both response inhibition, as in “punishment,” and response in- 
SUgation, as in "avoidance learning." But this interpretation involves, 
in addition to the concept of fear conditioning, the notion that. once 
fear is attached to a stimulus or set of stimuli, fear reduction. (or 
"relief" ) will reward and reinforce whatever behavior occurs in con- 
Junction with this reduction: this, in essence, is what may be termed 
secondary: reinforcement, £y pe-1. , 

i deeade after the publication of the paper cited at the outset of 
this chapter, Brown & Jacobs (1949) summarized this general point 
Of view in the following words: 


An important concept in à number of curr nt theories of behavior is that 
the emotion of fear is (usually) a learned, anticipatory response to painful 
Stimulation and that its significance as a botinan o Med ecd abire 
In its motivational properties. The assumption that fear is a learned response 


À : T ERD (C itioned ) stimuli which in 
Stems fro + fact that it can be elicited by (conditione 4 
m the fact th: or have been followed by, noxious 


the past ] < associated with, 

ast have been closely associated \ : QUT oxio 
(unconditioned). stimuli, The addirional as pire that ie ii Gus pro- 
duces) a drive, rests on the assumption that it exhibits certain of the major 
> attributed to primary drives such as hunger, 
is said to energize or motivate behav- 
lor, and (2) a reduction in fear is said to act as à ene for the ac- 
quisition of new responses. Since the fear reaction Is earned, the resulting 
: d to distinguish it from primary drives 
those typically observed in learning 


functional properties commonly : 
Specifically, (1) the presence of f 


“TIVE is termed secondary or acquire 
b hose antecedent conditions are unlike 
situations. 

bins conception, which 
D (Freud, 1936) into sa ide a 
Mowrer i Subsequently, he and others t l 1 
bites de nds of behavior and hay c inde UE ; ee T 
Periments tending to support the hypothesis a s os H E | rA i - 
ler & Dollard, 1941: Mowrcer, 194027 Mowrer Me e s aie j ads 
Study by NX, E. Miller (194822. V hich apparently pros aes. ie d cons inc- 
ing confirmatory evidence, forms the point of departure for the present in- 


Vesticar; * 
estigation Cp. 747A 


is essentially a translation of certain Freudian 
: esponse terms was first proposed. by 
applied the notion to the 


"Bee lve ination picture demonstration made by Miller & Hart. (1948). 
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Brown & Jacobs then briefly describe the experiment, discussed in 
Chapter 3. in which Miller trained rats to run from the white com- 
partment into a black onc. poow ipa » abe and then nie 
showed that the rats would learn to perform various Instrumental acts, 
under pressure of the fear drive, as a means of causing a door to open 
which initially blocked entrance into the black. or safety, compart- 
ment. Since the latter part of this experiment: involved "frustrating 
the animal (see Chapters 6 and 11) with respect to a "habit? previously 
learned, namely that of freely escaping from the white compartiment 
into the black one, Brown & Jacobs reasoned that: 


Although Miller's interpretation of his results in terms of fear and its re- 
duction appears quite plausible, there are other interpretations that merit 
consideration. One alternative is to assume that frustr 
important drive in the situation and hence that frustr: 
significant reinforcing event, According to this hypothesis, the interruption 
or blocking of any strong, on-going response produces a state of frustration 
or anger which functions as a drive. . . . This driv c, like that of fear, could 
lead to appearance of varied activity and, eventually, to the correct response 
of turning the whecl. Since the opening of the door would permit the re- 
sumption of the running response, there would follow an almost immediate 
reduction in frustration. As a consequence, the tendency to perform the 
wheel-turning response would be strengthened by this decrease. in drive 
(p. 748). P 


ation, not fear, is the 
ation-reduction is the 


Brown & Jacobs summarize their own experimentation and the re- 
sults thus: 

Two experiments have been described in w hich 
of the assumption that fear functions as 
havior and that fear-reduction serves 
of new responses. In both studies, the 
fear reaction and in measuring 


an investigation was made 
a drive to motivate or energize be- 
á reinforcing event in the learning 
s methods employed in establishing the 
ar rc i n Its reinforcing properties were designed to 
munimidzce the possibility of arousing some other emotional state ‘such as 


frustration or conflict to which the results might be attributed instead of to 
fear... S 


The general procedure of both studies involved 
during which rats were placed in an oblong box and given paired presenta- 
tions of a conditioned stimulus and an electric shock. Control animals. re- 
ceived the conditioned stimulus but not the shock. On silage uen learning 
trials the conditioned stimulus was presented alone to all anini and an op- 
portunity was provided for them to perform a new response of crossing from 
one side of the box to the other by jumping over a central barrier. When the 
new response was made, the conditioned stimulus w as immediately termi- 


a number of training trials 


nated. 

The assumptions underlying this procedure were ( 
entation of the conditioned stimulus and shock, 
escape response was systematically reinforced, 
ment of a conditioned fear reaction elicited 1 


a) that the paired pres- 
even though no specific 
would lead to the develop- 
PN the conditioned stimulus 
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alone: (bo that the conditioned fear would motivate the behavior of crc 
ff Reh een MER ae 3 

ng the barrier; and (c) that the reduction in fear, occ sioned by the c 
ion of the conditioned stimulus following the new response, would act to 


T the tendency to perform that response. 
Since in both experiments the fearful animals learned the new response, 


v hereas their comparable controls did not, the conclusion is drawn that fear- 
eduction functions much like other drive-reductions to reinforce new re- 


Sponses (pp. 7 


59). 
i In other words, during the original conditioning of fear to the 
signal (a high-pitched buzzer), there was nothing that the subjects 


could do to eliminate the fear (or avert the shock), so there were no 
N cll established fear-reducing "habits." Therefore, when the low bar- 
Mer was introduced betw een the two ends of the rectangular compart- 
ment, the rats were in no wav frustrated (as they might have been 
When Miller closed the door between the white and black compart- 
ments); and the motivation for learning to shuttle over the barrier 
When the buzzer was presented was clearly fear and fear alone. Brown 
& Jacobs did not, in their experiment, prove that. frustration. and 
i in the Miller experiment; they simply 
fractionated out the possible frustration-anger clement and showed 
that fear alone ca motivate, and its reduction reinforce, a response 
such as is involved in the shuttling, on signal, from one side of this 


apparatus to the other. 
19502, 1950b; Solomon & 


Numerous. other. writers. ( Lichtenstein. 
ar-rec 


anger were not operative 


luction conception of sec- 
ondary reinforcement and have reported a variety. of experimental 
; rse—as the reader will have noticed— 
is presented in Chapter 2 as most 
of punishment and avoidance 


Wynne, 1954). have discussed. the fe 


Confirmations. |t is, of cou 


ne conception that 
facts 
1 the immediately preceding pages 
iditional historical perspective and 
another conception of secondary 


exactly the 


adequately accounting for the 
learning, It has been reviewed ir 
Mainly as a means of providing ac 
Of contrasting and comparing it with 
reinforcement now to be considered.” 

the centrality of fear in both active and passive avoid- 
4 by studies involving the use of various 


ative of these studies is one recently reported 


Iy shown th 
à slightly. if at all, less so to a food 


but no less interesting. example, has 
Because of the known selective 


i Much new evidence for 

ATO" TAS 

: €e learning has recently 
n HEN es $ 
Nquilizers and sedatives. Represent 


b : . d 

Y Wenzel (igs8). Here it was clear 
a signal but only 
í arlier 


been accrued b 


at reserpine makes cats decidedly 


ate oe to a danger 
s al. And Conger (1981). 
“ported a similar finding fer 
inis of these and other chemica hr 

¥ convincing way, the thesis that fear 
behavior, Vhis, of course, was never doubtec 
ONY in a strictly behavioristic frame of reference. 


to take an € 
alcohol in rats. 


1 substances, such 


findings support, in a particu- 
is a decisive causal factor in avoidance 


I by common sense; “proof” is needed 
(See also Rutledge & Doty, 1957.) 
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Il. Secondary Reinforcement as 
f Conceived by Pavlov and Hull 


A good many years ago. Pavlov (1927) and his students reported 
a type of learning from which a different conception of secondary: 
reinforcement has evolved. As a means of illustrating “second-order, 
or “secondary,” conditioning, as he termed it. Pavlov describes an 
experiment ( performed by Frolov) in which the salivary reflex, first 
elicited (in a hungry dog) by food, was conditioned to the sound of 


a metronome (Fig. 4-3). Said Pavlov: 


Metronome ~ _ Fig. 4-3. First-order conditioning of salivation to 


Food ———= Salivation the sound of a metronome. The original, uncondi- 


doned stimulus for the salivation was food. 


Metronome Salivation 


[Then a] black square is held in front of the dog for ten seconds, and after 
an interval of fifteen seconds the metronome is sounded during 3o sec- 
onds. .. . 

Prior to these experiments the appearance of the black square had no sec- 
retory effect at all. As seen from the ... table | not reproduced | the con- 
ditioned reflex of the second order is measured even at this carly stage of its 
development | roth trial | by 51 drops... (p. 34). [See Fig. 4-4.] 


Fig. 4-4. Second-order conditioning of sal- 


Black aie a ivation to the sight of a black square. ‘The 


Reels "rn or first-order conditioned stim- 
Metronome — —— Salivation metronome, or first-order condition i 
ulus, here takes the place of the food as a 
means of eliciting the salivation in conjunc- 
tion with the black square. 


Black square —— — Salivation 


Pavlov then makes the following, more general comment: 


It was found impossible in the case of alimentary reflexes to press the sec- 
ondary conditioned stimulus into our service to help us in the establishment 
of a new conditioned stimulus of the third order. Conditioned reflexes of the 
third order can, however, be obtained with the 
conditioned reflexes in defense reactions such 
the skin by a strong electric current. B 


help of the second order of 
as that against stimulation of 


ut even in this case we cannot proceed 
further than a conditioned reflex of the third order (p. 34). 


For Pavlov, “reinforcement” occurred whenever a formerly neutral 


stimulus was followed, or “confirmed.” by an unconditioned stimulus 
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or by a first-order (or higher-order) conditioned stimulus. Hence, if 
a second-order CS were followed by a first-order CS, which now 
functioned as a reinforcing agent (in place of the original uncon- 
ditioned stimulus), it was logical to refer to this procedure as in- 
volving second-order, or “secondary.” reinforcement. For Pavlov, re- 
inforcement was, in any and all cases, a matter of pure contiguity, 
of a first-order CS occurring in conjunction with an UnCS, a second- 
with a first-order CS, or vet higher-order pairings. In first- 
in the illustration given, there is, in addition to 
4, the unconditioned 


order CS 
order conditioning, 
contiguity. of stimuli, also drive reduction; i. 
Tum occurs conjointly with the metronome but 


stimulus (food) not only 
Pavlov. disregarded the latter function of the 


av. as Hull subsequently did (see Chapter 3), 
1CS. while a necessary condition for 


also reduces hunger. 
UnCS: thus one could s 
that contiguity of CS and Ut 


first-order conditioning, is not a sufficient one; the reinforcement af- 


forded by drive reduction may 
But what is the situation as 
second order? Here salivation is or 
the metronome; and the clicking of the metronome could hardly be 
food and to reduce hunger (see Miles & 


said to be a substitute for 
Wickens, 1953. and Simons, Wickens, Brown, & Pennock, 1951, dis- 


cussed in Chapter 6). ‘Therefore, whatever the situation may be as far 
as first-order conditioning is concerned, we seem to have in second- 
a form of learning which occurs without drive 


also have to be present. 
regards salivary conditioning of the 
iginally evoked, not by food, but by 


order conditioning 
reduction, Here mere contiguity, 
and sufficient for learning. 
this problem, within the framework 


or proximity, of stimuli appears to 
be both necessary Let us now see how Hull 
approached and tried to handle 

drive-reduction theory. 
While rejecting certain fcatures of Pavlovian theory, Hull (Chapter 
3, p.e 68) was nevertheless. much influenced. by Pavlov's work on 
In the beginning of Chapter V Il of Prin- 


of a monistic, 


secondary reinforcement. 
ciples of behavior (1943) 
experiments on second-order saliv i 
"the metronome [in the demonstration previously described] had not 
only acquired thc capacity to evoke the flow of saliva but had also 
acquired the capacity itself to act as a reinforcing agent” (p. 85). 
That is to sav if. for example, after 
7 higher-order conditioning, it had been 


| Hull makes explicit reference to Pavlov’s 
ary conditioning and concludes that 


the metronome had become a CS 


and thus capable of mediating 
the subject m 
right, the tendency to make this movement 


hened merely by the presentation of the 


presented each time ade a particular movement, such as 


turning its head to the 
would have been strengt 
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metronome, without the accompanying presentation of food. From a 


common-sense standpoint this. of course, is not hard to understand: 


nee the Metronome was a stimulus which the hungry dog was "glad" 
to have occur (since it foreshadow ed the delivery of food), naturally 
the dog would learn any action which produced this stimulus. In a 
later book, Hull (1951) cites several laboratory. instances of this sort 
of thing: 


The click of the food-release mechanism in Skinner's experiment served as 
the [secondary | reinforcing agent there involved. The sight of the grains of 
boiled rice in Grindley’s experiment served to reinforce the locomotor re- 
action of his chicks even for a time when they received no food. The click 
of the food release mechanism in Bugelski's experiment vielded enough rein- 
forcement to delay the progress of experimental extinction, And ‘Cowles! 
chimpanzees recen ed secondary reinforcement from their possession of col- 
ored tokens which would later be inserted into a vending machine and ex- 
changed for a raisin. Miller (1951b) has given an elaborate tabulated analy- 
sis of a dozen or more additional cases of analogous nature. . . . Since this 
indirect power. of reinforcement is acquired through learning, it is called 
secondary reinforcement (pp. 27-28). E 


Thus we see that Pavlov and Hull were interested in two different 
* ^( ` " d B ` H Diy ye ?. c . 
aspects of the same phenomenon: Pavlov in the fact that a stimulus, 
once conditioned, could serve as the basis for still further (higher- 

MEO F E S 
order) conditioning; Hull in the fact that such a stimulus could be 
used to establish some entirely new “habit.” Hulls views in this con- 
nection, as of 1943. are summarized in the following brief paragraphs. 
which are found near the end of the chapter on secondary reinforce- 
ment in Principles of bebavior: i 

Our detailed findings concerning secondary reinforcement may be listed 
as follows: ^ : 

Ls Perhaps the most striking characteristic of secondary reinforcement is 
that it is itself a kind of by-product of the setting up of a recepror-etfector 
connection, in the first instance through primary reinforeement. Primary 
reinforcement, on the other hand, appears to be a native valearmed ea yacitV 
in some wav associated with need reduction. : Í E E 
EE Secondary reinforcement may be acquired by a stimulus from associa- 
tion with some previously established secondary reinforcement, as well as 
^ ith a primary reinforcement, It would appear that transfer of gk jawer üt 
reinforcement from one stimulus situation to another may go on inde&niraly. 


: T Ma E 
given the conditions of stable and consistent association. | Pavlov, 1927. 00 
the other hand, put a limit on this process (cf. p. o8). ] eae MNT 
3 A receptor-cffector connection | "habit"| involving any effector. may 
be reinforced by a secondary reinforcing situation E y d 
+ Secondary reinforcement differs from primary reinforcement in that 
the former iim to be associated, at least in a molar sense, with stimulation. 
whereas the latter seems to be associated with the cessation of stimulation. 
(p.97). E P 
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In what wav or ways, then, can the Pavlov-Hull conception of 
from the other phenomenon, 
an has also been applied? 


secondary reinforcement be said to ditfer 
previously cited, to which this same ter 
Secondary reinforcement type! refers quite simply. as we have seen, 
tisfaction experienced when a danger signal termi- 


to the reward or 
bjectionable event has 


nates, thus indicating that some impending o 
reinforcement £y pe-2 refers, it seems, 
enced when a stimulus comes 97. 


been averted; whereas secondary 
to the reward or satisfaction experi 


indicating the imminent occurrence 
secondary reinforcement. type-I im- 


reinforcement ty pe-2 


of some desired event. Or, in 


completely nontechnical terms, 


plies relief (withdrawl of treat); and secondary 
implies hope (presentation of a promise). 


In some wavs it is unfortunate that the same 
other ways it 1s perhaps 


term should have come 


into use for both these phenomena, but in 
an advantage, The common usage is certainly i i 
that both of these phenomena mediate learning without primary re- 
inforcement; but, over and beyond this, the: common usage also implies 


yeess OF mechan 


justifiable in the sense 


ism is the same, which may or 
who have been most in- 
have been explicit in their 
reduction in the 


that the underlying pr‘ 


may not be the case. Those investigators 


reinforcement 
ction is involved, viz. 
is Hull, as the principal spokesman for 
orcement, has taken the 


terested in tv pe-ti secondary 
x] . 


assumption that drive redu 
secondary drive of fear; where : 
those who have studied type- secondary reint : ' 
position (sec paragraph 4. pe 100) that this phenomenon involves stini- 


Wlation rather than “cessation of stimulation. The latter premise has 
rongly and explicitly by Seeman & Green- 


been stated even more st 


berg (1952), as follows: 


a highly significant construct 
in Hull's theoretical system. + + * It is Hull's view, ee that us fi t sec- 
ondary reinforcing stimulus acquires 1s power of reinforcement Y virtue 
g conie i ctional component of the need reduction 

occurrence, W herever it takes place, has a spc- 
a degree proportionate to the intensity of that 

». TOONS. 2 6% All writers agree, or have until 
ne property of the concept lies in the acquisi- 
bv an initially non-reinforcing stimulus in 
to à stimulus W hich is a primary re- 
] point here, then, is that, although 
with primary reinforcement, 


Vhe concept of secondary reinforcement ! 


of having conditioned to it 
"t the goal situation, W hose € 
cific power of reinforcement 1N 
occurrence” (Hull's Principles. 
recently agreed, that the definir 
tion. of. reinforcing properties by an 
consequence of is consistent. proximity 
Inforcer | drive reducer [.' The essentia 


: 3 2c operate jointly 
secondary reinforcement. n operate } ; 

aliv accurate: The use of the expression, secondary 
i econdary 
a with what Seeman and Greenberg 


toric; 

luction in a 5 

ee ir > of the term 

F as Ts : ils use t 

dard v rl e ate connotation (see, for example, \illercand Dollard, 
egiamd a 


aa : ; 
This statement is not his drive such as fear dates back 


reinforcement, to. denote Tes 


regard as its more 
1941). 
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t does, for ex xample, in Denny's (1948) study, it can operate in tbe absence 
ae a 

ee such primary reinforcement once it bas acquired secondary rein- 
fo i "ment properties. It can operate, that is, in the absence of any drive re- 
oret r 

duction of any kind (pp. 1-2). 


If one takes the position that secondary reinforcement type-2 does 


not involve drive reduction, it leads, as the following section P 
indicate, to a number of logical and psy chological dileimmas: and i 

one takes the reverse position, l.c. the view that secondary EA beg 
ment ty pe-2 involves drive reduction no less than does secondary re- 
inforcement type-1, there are important conceptual and empirical prob- 
lems, which will be dealt with, at some length, in Chapters 5 and 6. 


IIl. Paradoxes in Hull's Treatment of 
Secondary Reinforcement 


Hull has been more responsible than any other writer for stressing 
the general systematic importance of secondary reinforcement ty pe. 
vet his conceptualization of this phenomenon leaves much to be de- 
sired; and because secondary reinforcement is so central to the argu- 
ment developed in the present volume, it is essential that this phe- 
nomenon be understood as clearly and fully as possible. 


For Hull, second: 


ry reinforcement tvpe-2 was, in fact, the only 
form of secondary reinforcement, properly speaking, since he tended 
to lump secondary reinforcement type-1 (whenever he referred to it), 
along with primary reinforcement, under the more general heading 
of so-called. primitive reinforcement, i.c any and all forms of drive 
reduction, regardless of whether the drive was biologically given (as 
in hunger and thirst) or derived (as in the case of fear). What then. 
in Hulls own frame of reference. was "secondary. reinforcement? 
His formal statement on this score reads as follows: 


Corollary ii: A neutral receptor impulse [stimulus] which occurs repeat- 
edly and consistently in close conjunction with a reinforcing state of affairs, 


whether primary or secondary, will itself acquire the power of acting as a 
reinforcing agent (1952, p. 6). 


This formulation is probably valid (cf. Chapters 5 and 6); but it 
involves two logical difficulties as employed within Hulls own system. 


“Te will perhaps be helpful at this point to remind the reader that two-factor 
theory (second version, see C hapter 3) involved a clear understanding of what 


has here been referred to as type-1 secondary reinforcement. It was in respect to 


secondary reinforcement ty pe-2 that the theory was “silent” and therefore incom- 


plete and inadequate. It is, as we shall shortly sec, the systematic exploration of 
this short-coming that makes possible the major theoretical advance with which 
later chapters of this book are specifically concerned. 
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The first of these is that it is logically identical with Hull's statement 
of the conditions for the acquisition of secondary motivation or drive, 


namely: 


i: When neutral stimuli are repeatedly and consistently ass 


Corollary 
rv drive and this drive stimu- 


ated with the evocation of a primary or secondary 
lus undergoes an abrupt diminution, the hitherto neutral stimuli acquire the 
capacity to bring about the drive (Sv), which thereby become the con- 


dition (Ce) of a secondary drive or motivation (1952, p. 6). 


In effect, what the first of these formal statements says is that if a 
formerly neutral. stimulus occurs when some primary drive is re- 
duced, that stimulus takes on the capacity to act as a secondary rein- 
‘s is that if a formerly neutral 
is reduced, that stimulus 
ation. Now secondary 


forcer; and what the second statement say 
stimulus occurs when some primary drive 
takes on the capacity to arouse secondary motiv 
reinforcement and secondary motivation are, at the very least, different 
phenomena and, quite possibly. cven antithetical; yet Hull's two 
corollaries make them dependent upon identical conditions. Hilgard 


(1956), in discussing this paradox, says: 
ality of his exposition, Hull used many variations 
j ates, so that precise meanings are some- 
es in closely parallel 


Despite the seeming form 
in wording when stating his postul 
times hard to infer. | have restated the c $ 
form, which he did not do. Then it becomes evident that a neutral stimulus 
associated consistently with a reinforcing state of affair acquires two func- 
tions (and both at once! ): (1) the power to arouse a secondary drive (Corol- 
lary i), and (2) the power to reduce drive stimuli, and hence act as a sec- 
ondary reinforcing agent (Corollary 1). , fea 

.. The attempt to interpret the actual events in secondary reinforce- 
ment came late in Hull's theorizing, and he probably had not developed a 


theory fully satisfactory to himself (pp- 


two corolla 


129-130). 

This difficulty. arises because of Hull's reluctance to admit the cx- 
istence of two forms of primary reinforcement, namely what are 
and punishment or decremental reinforce- 
ental reinforcement (tvpe-l). The diff- 
we (secondary motivation) from dec- 


commonly termed reward 
ment (type-D) and incrementi 
culty: in deriving fear conditionir ELIO 
remental reinforcement has. been considered in Chapter 3 and need 
not be reviewed here. Suffice it to say that the contradiction implied 
by. Hulls Corollaries i and ii is readily and satisfactorily resolved by 
the view that there are two basic, and basically different, reinforce- 


rather than only one." 


ment processes, i . 
; difficulty in Hulls treatment of. secondary 


But there is another 


"For an attempt by Spence to justify Halls treatment of secondary reinforce- 
ment, see Chapter 8. p. 26! tf. But see also MeGuigan (1956) for further criticisms 


thereof. 
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reinforcement which is even more remarkable—and. unnecessary. As 
already. indicated (in the preceding section), Hull's formal SM stem 
implied that. secondary reinforcement involves an Herease im stimu- 
lation, as opposed to the decrease in stimulation (drive) which is in- 


volved in primary rcinfor ‘ 
most conspicuous of which was that it forced Hull's system bevond 
its avowed monism as regards the nature of reinforcement. As just 


ement. This led to numerous problems, the 


noted, Hull refused to recognize the reality of incremental reinforce- 
ment; vet in conceiving secondary reinforcement as he did, there was 
no alternative but to grant that reinforcement sometimes involves 
drive reduction and sometimes not. If a secondary reinforcing agent 
can nevertheless mediate both higher-order conditioning and new habit 
formation, then the only principle of reinforcement which is left here 
is that of classical Pavlovian conditioning, namely, the mere temporal 
conjunction or “contiguity” of two stimuli, 

It is true that Hull vacillated on this score. For example, at the end 
of the chapter on secondary reinforcement in his Principles of Be- 
havior (1943), there is a long note entitled: “Are Primary and Second- 
ary Reinforcement at Bottom Two Vhings or One? 


And here he says: 


So far as our present know ledge goes, the habit structures mediated by the 
two types of reinforcement agents are qualitatively identical. This considera- 
tion alone constitutes a very considerable presumption in favor of the view 
that both forms are at bottom, i.c., physiologically, the same. It is difficult 
to believe that the processes of organic evolution would generate two en- 
tirelv. distinct physiological mechanisms which would vield qualitatively 
exactly the same product, even though real duplications of other phy siologi- 
cal functions are known to have evolved (pp. 99-100). 


It is true that later, as Hilgard ( 1956) has noted, [lull “was attempt- 


ing to hold a consistent drivc-stimulus reduction theory of secondary 
reinforcement (as well as of primary. reinforcement)” (p. 130), 35 
indicated by the following statement: 


It follows that any stimulus consistently 


associated with a reinforcement 
situation will through that association 


tu WAU Che acquire the power of evoking the con- 
ditioned inhibition | of a driv ¢-producing response such as fear |, i.c. a reduc- 
tion in stimulus intensity, and so of itsclf producing the resulting reinforce- 
ment. Since this indirect power of reinforcement is acquired through learn- 
ing, it is called secondary reinforcement (Hull, 1951, pp. 27-28). 


In subsequent chapters we shall see that the idea 


which Hull was 
exploring in the above quotation h 


as considerable promise and. powers 
but it is not the one which his followers have 


most generally accepted 
in this connection. Instead they 


“have seized upon and utilized his con- 
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jecture that secondary reinforcement is mediated by a fractional an- 
ticipatory goal reaction, r,, whose occurrence produces a distinctive 
stimulus, s, Which, in some wav, carries the acquired, or secondary, 
reinforcement potential. This notion was stated differently from time 
to time by Hull; but the following formulation is selected here from 
the Principles of Behavior because it occurs in the same note, already 
deed. the possibility that primary and sec- 
ondary reinforcement are "basically the same, Hull goes on to observe 
that since “the present development of neurophysiology is quite remote 

ney of the problem calls for “an 
` after which we read: 


cited. Having acknowle 


from such an achievement,” the urge 
attempt at a workable first approximation. 
_ These considerations suggest rather strongly that the first secondary rein- 
forcing stimulus acquir “its power of reinforcement by virtue of having 
conditioned to it some fr ctional component of the need reduction process 
Of the goal situation (G) whose occurrence, never it takes place, has a 
Specific power of reinforcement in a degree proportionate to the intensity 
Of that occurrence. 

of the goal reaction by the 


let us represent this fractional component 
symbol g [or ,] (1943. pe 100). 


The following excerpt from K. W. Spence’s recent Silliman Lec- 
tures at Yale University will provide 
been made 


one of many possible illustrations 


Of the widespread use that has of this concept: 
The essential notion of this formulation . «+ 1S that learning in instru- 
ces involves classical conditioning of the goal or con- 
lus cues throughout the chain as well as an in- 
c instrumental acts themselves. There is, 
f the conditioned fractional anti- 
duced the concept in his early 
1931), although he never des 
1 conditioning. The major dif- 
his theoretical mechanism and mine is that 
ator and have identified it with Hull's 
al construct. (Spence, 19512, 


Mental reward sequen 
Summatory response to stimu 
crease in the habit strengths of the in 
bout this notion o 
Indeed. Hull intro 
aze learning (1930+ 
e instrumenta 


of course, nothing new a 
Cipatory goal response. 
theoretical articles on m 
Cribed its possible role in simpl 
ference between. Hull's use of t | 
l have regarded it primarily as a motivator an 
more recently formulated incentive motivation 
t951b) (Spence, 1956, p. 126). 


a theory of “habit” which will 


to evolve 
rent in the S—R "bond" con- 


Here Spence is trying j 
not be beset with the difficulties inher 
Ception thereof. an objective which is o ; 
the present stage of learn (sce Chapter ‘ 
Means certain that the particular approach suggested in the foregoing 
quotation is the most promising is (ch: Grapsi & aid. BY; In the 

formulation will be suggested, on 


f the upmost importance at 
but it is by no 


ing theory 


f : spnative 
allowing chapter an alternative | : : 
of this volume will depend; but first 


Which the validity of the rest 
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it will be desirable to take a more detailed look at secondary rcin- 
forcement itself, not logically but empirically. If, after all, there 
should be any question about its reality, qua phenomenon, there 
would be little point in belaboring its theoretical significance. 


IV. Experiments Exemplifying Secondary 
Reinforcement Type-2 


As will be seen in Chapter 10, there were a number of carly experi- 
ments on delayed primary reward which, inadvertently, involved the 


operation of secondary reinforcement tvpe-2. But here we shall con- 


sider instead some of the outstanding attempts which were made to 
investigate this phenomenon as such. One of the earliest and best of 
these was reported by B. F. Skinner in 1938. After alluding to Pav- 
conditioning (which Skinner 
termed “secondary conditioning of ‘Type S," to stress the stimulus or 
sensory elements in the situation), this writer went on to say: 


lovian higher-order, or “secondar 


There is, however, a process that might be called secondary conditioning 
Type R | for response |. It does not involve a conflict with the process of 
discrimination because it is a response rather than a stimulus that is rein- 
forced. The process is that of adding an initial member to a chain of reflexes 
without ultimately reinforcing the chain. In the present example, the sound 
of the magazine acquires reinforcing value through its correlation with ulti- 
mate [primary] reinforcement. It can function as a reinforcing agent even 
when this ultimate reinforcement is lacking. Its reinforcing power will be 
weakened through the resulting extinction, but considerable conditioning 
can be effected before a state of more or less complete extinction is reached. 
The maximal result to be obtained from a given amount of previous con- 
ditioning of the sound of the magazine is shown in the fo lowing experi- 
ment. 
y The usual preliminary procedure |preliminary, that is, to the rat's learn- 
ing to press a bar as a means of obtaining food] was carried out with four 
rats, as the result of which they came to respond readily to the sound of the 
magazine by approaching the food trav. Sixty. combined presentations of 
the sound and food were given. On the dav òf conditioning the magazine 
was connected with the lever but was empty. For the first time in the history 
of the rat the movement of the lever produced the (hitherto always rein- 
forced) sound of the magazine, but no responses were ultimately reinforced 
with food. The four resulting curves are given [in Fig. 4-5] (Skinner, 1938. 
p- 82). 


In going over the galley proof of this book with a class in the 
summer of 1959. the author was interrupted at this point by a student 
who asked a very pertinent question, one which indicated that she was 
beginning to grasp and apply the general theoretical system which is 
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here evolving. In. paraphrase, what she said went something like this. 
During the training stage of the experiment just described, Professor 
Skinner periodically pushed a key which caused the food delivery 
mechanism to click and to deliver to a hungry rat a pellet of food. And 
when the experimenter's key was replaced by one (the bar) which the 
rat could operate, the rat proceeded to push it and make the delivery 
mechanism operate several (about 50) times, even though it was now 
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Fig. 4-5. Cumulative curves of bar pressing obtained by Skinner (1938, p. 83) 
reinforcement only. Phe latter consisted 


from four rats which received secondary ; t 
of the sound produced by the operation of a (now empty) food-delivery mecha- 
1 associated 6o times with the delivery of food. 


nism. Previously this sound had been 
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completely empty. In other w ords, all the rat got for his efforts, during 
this testing phase of the experiment, was the click. Now, said the stu- 
dent, let us train a rat in a different w av. 
click and food associated 60 times during the t 


in wc are going to have 
aining period; but instead 
of Professor Skinner pushing the key that makes the food-delivery 
mechanism operate, we will require the rat to do it. In short, wew ill set 
up an ordinary bar-pressing habit, with 60 rewarded training trials. 
Now, in the test period (when bar pressing still produces the click but 
no food), how many times will this rat press the 
times than will a rat trained in the other w ay? 
Immediately another student rephrased the question, 
words, both training procedures will cause the click 
ondary-reinforcement properties—which 
one procedure (arbitrary pairing 


bar? Fewer or more 


thus: In other 
to take on sec- 
will give it the most? The 

of click and food) involves the 
the other procedure is clearly in- 
strumental (click is followed by food only if the 
question is, Does the one procedure work b 
the other? 

Here still another student entere 


classical conditioning procedure; 


bar is pushed). So the 
etter, in this situation, than 


d the discussion and said she thought 
the second rat would press the bar more often following withdrawal of 


primary reinforcement than would the first. And the reason she gave 
was this. The rat for which food follows click (during training) only 
if the bar is pressed will be producing alot of other stimuli besides the 
click (sight and touch of the bar, proprioception, ete.) which will 
acquire secondary-reinforcement properties (or Sr, 
happily introduced by Skinner), where 
click will be the Only stimulus produced by bar 

have St properties at the beginning of the test (or 
This inference has far-r aching implications, which 
claborated until Chapter 7. But the reader can use it here to test himself 
on how well he has assimilated the discussion up to this point. 

Then, observed a fourth student, one could do something else. One 
could test the strength or resistance TO extinction. of a bar-pressing 
habit under two different conditions: 
food, and extinction with neither 


to use a notation 
as in the casc of the other rat the 
pressing which will 
extinction) period. 
Will not be further 


extinction with click and no 
click nor food. ‘This procedure, he 
potency had been acquired by the 
other Tesponse-correlated s 
to the experiment which « a 
immediately: after 


said, would show us how much S: 
click and how much by the 
gestion now leads us back 
next in the galley proof, 
Skinner experiment, 
About the same time. Bugel 


tumuli. This sug- 
s to be discussed 
the description of the 1938 


ski (1938) demonstrated secondary rein- 
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forcement in a related wav, by show ing that, following acquisition of 
a bar-pressing habit with regular primary reinforcement, rats will con- 
Unue to press the bar Jonger, after the withdrawal of primary reinforce- 
ment, if the bar pressing continues to activate the (now empty) food- 
delivery: mechanism and thus makes the accustomed sound, than if 
the bar and delivery. mechanism are disconnected. Here we see that a 
stimulus which is produced by the subject’s own actions and is then 
followed by primary reinforcement takes on secondary reinforcement 
Capacity, just as does a stimulus which is produced by the experi- 
menter without reference to the subject’s behavior, as in the example 
quoted from Skinner? And this same general type of phenomenon was 
further exhibited in studies by Wolfe (1936) and Cowles (1937). 


in Which it was shown that if a chimpanzee learns to obtain food 
by Inserting poker chips into a sort of “vending machine,” the chips 
themselves take on unmistakable secondary reward value, as indicated 
by the subject's readiness to "work" to obtain them. 

Earlier it had been shown by Katherine Williams (1924) that a 
Small box. from which food had been repeatedly received by rats in 
a different situation could, it placed in the goal compartment of a 


T : : fiere his experiment v 
Maze, be used to fixate performance therein. But t I 
re subjects had been allowed to explore 


Complicated by the fact that tl : a 
the maze extensively. before the empty food box was introduced, so 
that the procedure was more 


like that employed in latent-learning 


Nperiments involving primary reward (sec Mawr; ier Chus 2) 
than like the conventional type of maze-Iearning procedure. There- 
fore, in 1949, Saltzman reported a study 
this comment: 


which he introduced with 


of secondary reinforcement learning in the 
first to establish the fact that rats can 


In order to study the topic 
reward. Williams! success 


Maze situation, it seemed advisable 


Ee a ae 

solve a qm; 5 he basis of secondary 
a maze problem on the Das » vea. WT fe ua 

With a EROR Ne ETE technique suggests, but does not prove, that rats are 


able to do so, She points out that w hen a con uc reat ey 
Was used the rats did not show any improvement wy qug ata in ds 
Studies by Anderson ( 19412) on perci qa : € zm “i Š: a 3l 
offer evidence that such learning can occur. I as d 5 e - ure a the sec- 
ondary reinforcement in these studies is not clearly defined (p. 162). 


be noted between : ) 
avoidance learning, as previously: considered. 


UD lel iya these two examples of secondary 
We parallelism shoule P 
reinforcement and active and passive 
In the Skinner example. gue id 
"üt, experi roduced. stimulus, 
` Te nter-produced * By Reon ct aan M uu uu 
ilies d ERST iski example secondary reinforcement. is conditioned to a 
"ls, ` gelsk i x: s e X Sans: i n 
BE Pag ated stimulus, just as fear is in passive avoid- 


Fesponse-dependent, response-c arrel 


inforcement is conditioned to an independ- 
just as fear is in active avoidance learning; 


ance learning, 
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What Saltzman did, in essence, was to teach rats to travel down a 


black) goal compartment, entry into which also involved distinctive 
un M * 

kinesthetic stimulation. Then, in a modified T-mazce. 
k 


hort, straight alley to obtain food in a distinctively painted (c. g. 
g 3 j 


he tested. the 
capacity of this compartment, at the end of one wing of 


to cause the rats to go to it instead of to a visually: and. kine 


different compartment at the end of the other wine of the maze. 


studies that. secondar M 


so he tried a variety. of pro- 
cedures aimed at increasing its stability. and durability. His 


the maze, 
sthetically 


But Saltzman knew, of course. from earlier 
reinforcement tends to be evanescent, 


investi- 
first of which involved 
the use of four groups of rats. Group 1 Was given " 


consecutive rein- 
forcement" in the runway tr aining. i.c., on cach s every entry into 


the goal compartment, the subjects found food. 


gation was divided into two major parts, the 


Group 2 Was given 


"a rewarded trial always 
preceded and always followed a non-rewarded trial” (p. 164). And 


Group 3 was given “differential reinforcement," as follows: 


“alternative reinforcement,” mca ining that 


The rats in this group received the same treatment as those in Group 2. 
with the exception that on the non-rewarded trials, the goal box was nor the 
same one that was used for the rewarded trials. 
the black goal box, found the white go: 
the non-rewarded runs, and those rece 
the black box on the non-rewarded runs 


The rats receiving food in 
| box at the end of the runway on 


iving food in the w hite box, found 
s (p. 164). 


In other words, the Group- 3 procedure was one 


(and kinesthetic properties) of the goal con 
always the same and therefore ambiguous (implying reward on some 


trials and nonrew ard on others), was Consistently: correlated with the 
presence or absence of food on B given trial. 


A control group of animals, Group G. 
training as did the Group-1 animals; but w hen tested in the T-maze 
the Group- C animals continued to receive primary 

(food) for the correct choice. In contr 
three (experimental ) groups 
“satisfaction” of entering the 


in which the color 
partment, instead of being 


received the same runway 


reinforcement 


ast to this procedure, the other 


got no food in the T-maze, only the 


goal compartment in which 
previously received food, in the straight alley. 

All animals received 1s runs in the 
were operating, they would on the 
right and 7.5 turns to the left. in a The animals 
Group 1 went to the correct side 8.3 


3 times; those in Group Bx 9 
times; and those in Group i. 10.7 times, Considering that the 


animals 
reinforcement in the T-maze (Group C) 


they had 


T-maze, so that, if chance alone 


average make -.s turns to the 
random order, 


which received primary 
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made only 10 correct choices out of the possible 15, the secondary- 
reinforcement groups made a surprisingly good showing. In fact, the 
Groupe; animals were actually superior to the Group-C animals 
(10.7 compared with 10.0) by an amount statistically significant at 
the o1 level of confidence; and all groups were significantly different 
(at the .os level of contidence or better) from the hypothetical chance 
expectation of 7.5. 

Thus we see that even continuous primary 
jects in the straight alley led to a statistically reliable secondary rein- 
forcing etfect in the F-maze, that alternate (partial) reinforcement 
increased this effect, and that so-called differential reinforcement. pro- 
all, even exceeding that obtained from the use 
Said Saltzman of this part of his investi- 


reinforcement of sub- 


duced the best result of 
of primary reinforcement. 
gation: 

ability of the rat to learn a simple maze 
d only by stimuli previously associated 
sossibilities of secondary reinforce- 


The experiment establishes the 
when correct choices are followe 
with food, and probably enhances the I DOREM T 
ment as a general principle of learning. Inasmuch as the distinctive aspects 
of the goal boxes were at least several, including kinesthetic as well as v sual 
components, it is impossible to say W hich of them w = the Most effec tive. 
Taken together, however, they were at least as effectiv e, or slightly more 
effective than food. This last fact challenges a re-interpretation of the whole 
effectiveness of rewards in learning (pp. 168-170). 


problem of the relative 

‘The second part of Saltzman’s study. being less directly relevant 
to our present concerns, can be described very briefly. Its purpose 
Was given as follows: 

If the failure to show [highly efficient] le: ming W ith the Consecutive Re- 
inforcement Method was due, as was suggested in Part 1, to the extinction 
of the acquired reward value, and not to Its absence altogether, then by 
merely preventing or retarding the extinction during the actual maze learn- 
ing, we should b able to show that learning can occur with this method. 
The procedure that was employed for retarding the extinction was one of 
interpolating rewarded runway trials between the maze trials during the 
learning tests (p. 169). 
act replication of Part | as far as runway training 
one group received consecutive reinforcement. 


Part IE was an ex 
Was concerned, i.c. 


one partial (alternate 


) reinforcement, and one so-called differential 
The difference came in the T-maze test procedure: 


reinforcement. 
“The procedure 
ach of the maze trials, except the last one, w 
169). Phe result was a dramatic 


differed from that used in the first study in only 
one respect: e: as followed 
by a rewarded runway trial (p. 
increase in the effectiveness of the runway goal compartment as a 


112 LEARNING THEORY AND BEHAVIOR m m 


ndary reinforcer in the T-maze situation: the Group-1 animals 
seco ary 


now making 11.5 out of 15 possible correct choices; the Group-2 


anim S Š i p pu 
11.6 correct choices—all of which exceed the performance of Group 


als making 10.4 correct choices; and the Group-3 animals making 


C (the same in both Part I and Part Il) of only 10 correct. choices 
out of a possible i5. Said Saltzman: "The effect of the interpolated 
runway trials [which involve primary reinforcement] is to prolong 
the effectiveness of the secondary reinforcing agent, and thereby make 
possible a successful measurement of learning" (p. ). 

Saltzman's over-all conclusions run as follows: 


1. Rats are able to learn a simple maze when the only reward is an ac- 
quired or secondary onc. " : 

2. Secondary reward may be as effective as the primary reward of food 
in influencing the choices made in a maze. 

3. Of the three methods studied for establishing secondary reward value, 
the most effective was the differential rewarding of positive and negative 
cues. 

+ The effectiveness of acquired rewards for promoting learning may be 
prolonged by continuing the process of establishing reward value during 
the actual learning tests (p. 173). 


One of the most significant things about the Saltzman experiments 
is that they not only give clear indication of the reality of secondary 
reinforcement, type-2, but that they also suggest an affinity between 
this phenomenon and “habit.” As will be shown later, a so-called habit 
proves more resistant to extinction (Chapter 11) if it has been rein- 
forced intermittently rather than continuously (Chapter 12); and the 


Saltzman results show a similar effect with respect to the acquisition 
by previously neutral stimuli of the capacity to act 


À secondary rein- 
forcers. As will later be indicated (Chapter 7), this similarity. has far- 
reaching implications; but these cannot be confidently developed until 
there is further exploration of secondary reinforcement qua phe- 
nomenon. This task will continue to occupy us in this and the im- 
mediately succeeding chapters. i 


"That Hull had no intimation, as of 1943. Of the possible 
nomenon called secondary reinforcement and “habit” is suggested by the fact 
that, although he knew that a habit can be made more resistant to extinction 
if established by means of intermittent reinforcement (ps 33735 i 


identity of the phe- 


€ thought that 


the most. effective secondary. reinforcement. was established by associating the 


secondary-reinforcement stimulus “repeatedly and | consistently" with primary 
reinforcement (Corollary ii, quoted on p. 102). However, Hull, w riting in 1951. 
showed a very good appreciation of the other view. He said, 


“The strength 
of a secondar 


y reinforcing stimulus developing in essentially the same manner 


that the strength of a habit Ebo develops might be a possible explanation of 
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V. Secondary Reinforcement Type-2, 
Doubt and Confirmation 


al evidence for the kind of secondary 


Despite the very substanti 
misgivings concerning its reality 


reinforcement here under discussion, 
airly prevalent. For example, Spence (1956) 
have expressed doubt concerning its 
logy; and Razran (1955), in a paper 
;onditioning—and Secondary Rein- 


and importance are still f 
and McClelland et al. (1953) 
systematic significance in psycho 
entitled “A Note on Second-Order C 
forcement," has recently written as follows: 


Now, if second-order conditioning is difficult to obtain and maintain in 
“ms to need cognition to be set up in human beings, what 
1 . conceived independently: as the ca- 
or less permanently another reaction? 


animals, and it se 
about secondary reinforcement: per 


pacity of a stimulus to reinforce more 
In the writer's opinion, the answer 1s essentially the same. True, the labora- 


tory evidence for secondary reinforcement in animals—while never too 
critically: appr ised—seems, to date, to be more consistent than that for sec- 
ond-order conditioning. Yet, as an isolated force, secondary reinforcement 
is, too, as any mere inspection of the data of its several experiments reveals, 

1 v extinguishable, and quite ancillary and immediately 
rv reinforcement. Thus there is here, too, a far cry 
ined (several experiments) to the. phenomena (the 
al behavior) which are purported to be explained— 
lementary accounts (p. 330). 


very impermanent, ver 
dependent upon prit 
from the results obt: 
vast area of man's cultur 
and lots of room is left for supp 
just completed 

ar towar 
beyond peradventure, the empirical 
Although somewhat com- 


Fortunately, work by D. W. Zimmerman, at the 
University of Illinois, goes f 1 dispelling doubts of the kind 
and establishing. 
of this phenomenon. 
employed by Zimmerman is. in 


sections of this chapter indicate, 


here expressed 
Validity and potency 
plicated in detail, the procedure 
principle, simple. As the preceding 
the study of secondary reinforcement (tvpe-2) normally involves 

: learning procedures, (a) association of a 
is with the occurrence of some form of primary 
stablishment of some new form of bc- 


two successive OE SABE 


formerly neutral stimult 
reinforcement and (b) the ¢ 


the operation of secondary reinforcement; and the same writer observes that 

nforcement s 
han consecutive 
of this surmise. While Hull was almost certainly 
lationship between secondary reinforcement aid 


ld seem that it is secondary. reinforcement. that 


"intermittent. primary rei hould increase the strength of a secondary 


more t primary reinforcement (ps 247) and 


reinforcing stimulus 


cites Saltzman (1949) In support 
an intimate TC 


ibit, it Ww ou 
han the other way 


right in suggesting 
the phenomenon of hà 


er up. s "n (see Chanter s 
explains babit, rather t round (see Chapter >). 
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kvot (habit) on the basis of the acquired, or secondary, reinforce- 
way : E x ri E 
nt now provided by the formerly neutral stimulus, alone. As al- 
mec x : " d n 
eady noted, Saltzman has shown that a formerly neutral stimulus (oi 
reads 3 $ 


stimulus compound) acquires greater secondary-reinforcement capa- 
city if associated with primary reinforcement intermittently rather 
than unfailingly; and various other workers (see Chapter 12) have 
shown that habits which are established with primary reinforcement 
are likewise more resistant to extinction if acquired on the basis of 


intermittent rather than invariable reinforcement. But apparently. no 
one, prior to Zimmerman, had thought of the possibility of using a 
double intermittent-reinforcement procedure, i.e., intermittent associa- 
tion of the formerly neutral stimulus with primary reinforcement and, 
subsequently, intermittent presentation of this stimulus upon the occur- 
rence of the, so to Sav, test response or new “habit.” Because the 
potency of secondary reinforcers had, at best, seemed slight and very 
susceptible to extinction, carlier workers were impelled to give the 
secondary reinforcer for cach and every occurrence of the test re- 
sponse. Bur this, reasoned Zimmerman, only “uses up" the capacity of 
the secondary reinforcer that much faster, so that. the economical 
and effective thing to do is to employ the secondary reinforcer 
sparingly, i.c. intermittently, The result of thus combining the ad- 
vantages of intermittenc 


v. at both the primary and secondary levels 
of reinforcement, has been to create a sort of multiplicative rather than 
merely summative effect. That is to sav, by means of the double 
intermittent-reinforcement procedure, Zimmerman has succeeded. in 
obtaining secondary reinforcement effects that are rot 
as great as those previously reported. His procedure is exemplified in 
a study recently: reported in the Ps) chological Review. Because of 
the special importance and novelty of this paper, 
tensively from a section thereof entitled, “A 
Reinforcement Procedure which leads to 
forcement.” 


iwhly 40 times 


we shall quote ex- 
‘| Wo-Stage Intermittent 
powerful secondary rein- 


With the experimental procedure to be reported here, it has been found 
possible to give a secondary reinforcer such strength that it could, if desired, 
be used as a class demonstration of the Law of Effect, 
of such primary reinforcers as food and w ater. TI 
stable secondary reinforcement can be established i 


much in the manner 
ae method whereby such 


s as follows: 
“Asa partial qualification of the above statement, it should be noted that Clav- 
ton, in 1952, had reported use of double intermittent reinforcement, 


but because 
she had emploved only very low ratios of reinforcement 


\ nonrcinforcement, this 
investigator failed to obtain the dramatic effects reported by Zimmerman, 


(Sce also 
Kelleher, 1957a. 1957b.) 
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A. Training. A thirsty: rat is put into a small (117 x 11” X 11") box fitted 
with a motor-driven water-delivery mechanism, After the animal becomes 
habituated to the situation, a formerly neutral stimulus is given secondary: 
reinforcement potential in this wav: A two-second buzzer, follow ed by op- 
eration of the water delivery dipper, is presented ar approximately one- 
minute intervals. Buzzer presentations. are programmed independently of 
the behavior of the animal with the exception that the buzzer never sounds 
While the animal hovers over the w ater-delivery aperture. 

After the approach response has been firmly established, water reinforce- 
ments are omitted follow ing the buzzer, at first on alternate presentations, 
then. successively in longer runs. The ratio of r warded to non-rew arded 
presentations varies somewhat randomly, the m number of reinforce- 
ag upward. A 1:10 ratio is finally stabilized, 
1 run being 1:14. What ‘is involved 
d to non-rewarded Sr (sec- 
his ratio being gradually. in- 


ment omissions, how ever, movir 
With the longest single. non-reinforced 
here, then, iş; variable ratio of w ater-rew arde 
ondarv-reinforcement stimulus) presentations, t ! i 
creased, During this training the vigor with which the animal leaps to the 
y aperture following the buzzer does not lessen, In spite of 
the frequency of “pay off" is progressively. decreasing. 

aining what happens is that the buzzer becomes a 
7 " Water is received at no time ex- 


Water deliver 
the fact thar 
During this type of tr 


signal “that water may now be available, ra Me 
cept ù n tbe buzzer sounds, so the animal soon learns to pay attention" to 


the Duzzer—to, as ic w ere, "jump at the chance” which it affords. But the 


buzzer is not a sure sign of water 
B. Testing. Follow ing the tr: 
is tested for its secondary reinfo a te 
Sequence of the rat's pressing a small bar s hich is noy Wenden 
zer (8") alone is thus used to strengthen this response, no further water rein- 
foreements being given. However, instead of having the buzzer invariably: 
; cents being g rocedure again involves intermittent. reinforcc- 
follow: bar presiis the pragen inm iled procedure that may be 
Ment. Here various options regarding the detailed procedure th: av be 
ised 5 E S es experimenter. The one that produced the results shown 
NEC "e oper > Un Pas camer a Sain, 
lin Fig Tl went as follows. On the first day of testing, Ps b Pressing, 
beh; aes as established by reinforcing (with the buzzer ) the first six re- 
" AUN É \) Thereafter Sr was given following a response only 
)onses (segment A). : " DOMENUS RTT SPE ees ni 
ws a f st a minute had elapsed since the preceding reinforcement (seg 
ment s ERO the buzzer was discontinued entirely, à typical extinction 
1 ; en the ó » UNA "condarv rein. 
perf ) f lowed (segment C). Reconditioning, with secondary rein 
T'ormancc foll ME 3 1 interveni Fliimar 
for € ly, was possible on a second day, without debian: pripa 
cement only, was pos DN aD sans TuS 
reinforcement of. the secondary. reinforcer, \ NE mopa hedules 
aes Factory. results (Zimmerman, 1957a) (pp. 373-374). 


ining procedure just described, the buzzer 
S senting it as a con- 
"ailable. The buz- 


reing properties by pri 


have also given satisfy 
A] > results thus obtained by Zimmerman are dramatic 
Although the results l PENNE a th. 
Enough, he has since devised a procedu à ig | 
Pan Ad f this procedure.” he says. "jn. 
are still striking. "By means of this procedure," he savs, “in 
H 3 OLE i ang. À Nob n : ae 
str iy y ior which appears to be virtually Mextinguishable can 
"mental behavior ap BN 


) blished the basis of secondary reinforcement alone (1957b 
© established on the basi: 


P- 1). The details are presented as follows: 
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beains; but w ith respect to the running behavior the procedure at this point 
becomes one of extinction. Bur because of the previous intermittent-rein- 
forcement training, the non-reinforcement condition which now holds with 
regard to running is not clearly demarcated, for the rat, from the foregoing 
acquisition. In common sense terms, we would thus say that the animal is 
“fooled” or “tricked” into working for a chance to execute behavior that is 
actually futile [see Chapter 12]. . 

As soon as the bar-pressing behavior “catches hold"—after five or six runs 
—it, in turn, is only intermittently reinforced (secondarily). This further 
prolongs the effect. The final stable performance which is obtained, then, is 
this: the animal presses the bar in the starting boy over and ove agai ¿and 
occasionally the gate lifts, releasing a run toward the empty goal box. Then 
the animal is returned to the starting box again, and the cycle is repeated 
(Zimmerman, 1957b, pp. 174). 


Preliminary results with this technique show that animals can be 


made to emit as many as 2.000 bar-pressing responses, with second: e 
reinforcement. only, and that even then the buzzer and gate raising 
have not entirely lost. their. secondary-reinforcement potency. This 
figure, of 2,000-plus responses, is to bc compared with the 1s L-maze 
runs reported by Saltzman and the 30 to 60 bar pressings obtained by 
Skinner (Section IV). There thus remains no question that secondary 
reinforcement, given the proper conditions, can be exceedingly per- 
sistent and powerful (sce also Zimmerman, 1958, 1959).!! This fact is of 
crucial importance for the on-going argument of this volume as a 
whole; but before that argument can be systematically developed, it 
will be necessary, in the two succeeding chapters, to assimilate some of 
the more immediate implications thereof. 


arousing) capacity, which persists for a time after primary reinforcement has 
been withdrawn. From this the notion, which is to be developed more fully 
in Chapter 7 


it follows that "habit strength" is secondary reinforcement, and not 
à connection or bond of the type posited by Thorndike and Hull. The genius 
of the Zimmerman experiment, it may be added, lies in its use of intermittent 
reinforcement (at both the primary and secondary levels) as a means of keeping 
secondary reinforcement (hope) from extinguishing (see Chapters i1 and 12). 
“In the passage previously cited (p. 113), Razran has held that the evidence, 
as of 1955. suggested thar second-order conditioning (as conceived by Pavlov) 
and for secondary reinforcement (as conceived by Hull) was equally sparce and 
unimpressive. Since a method has now been reported for establishing extremely 
powerful secondary reinforcement, the interestit È 


possibility is opened up that, 
by the same basic methodology (double” intermittent reinforcement), higher- 
order conditioning could also be made more effective, iic. capable of being carried 


to “orders.” or levels, well bevond the second Cor, in the case of “defense” condi- 
tioning, the third) order. 


This is a possibility which would. seem to warrant 
Immediate investigation. 
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VI. Special Implications of the Experiments of 
Culler and His Students 


‘Too late to be systematically integrated into this chapter, a number 
of highly pertinent observations have been discovered in a paper pub- 
Her. These will be presented in the order 


lished in 1941 by Fecher & Cu 
authors and will be articulated with 


in which they are mentioned by the 
various aspects of the argument which has just been concluded. First 


of all, Eecher & Culler sav: 


Finch and Culler (1934), using paw-v rdrawal in dogs as the response. 
succeeded in conditioning to it stimuli of first, second, third, fourth, and fifth 
sively by the simple expedient of apply ing a shock to the left 
c ailed to respond. hey ascribed the failure 

and of Foursikov (Pavlov, 1927, p. 34) to 
adequate drive or activation 


order succe 
thorax whenever the subject f 
of Frolov (Pavlov, 1927. Pe 33 
surmount the second or third order : 


of the animal (p. 223)- 


is due to in 


This excerpt bears upon the phenomenon of higher-order condition- 
ing (discussed in Section ID) and throws further doubt upon the argu- 
ment that classical conditioning theory can be made to account for 
Active avoidance learning (Chapters Iland IIL). Here is active avoidance 
d thout any recourse W hatever to the specific, 
namely, shock to the foot or 


learning, long maintained, wi 
unconditioned stimulus for 


paw. However, the stimulus actually va à 
) is an adequate (unconditioned) stimulus 


and Finch & Culler show that by 
keep their subjects responding to 


lcg flexion, 
used (namely, shock to the “left 


thorax" or lateral chest area 
for the emotional response of fears 
keeping it alive, they were able to bjec 
various formerly indifferent. (conditioned) stimuli without (after ini- 


tial conditioning) subsequently 
d ir 


presenting shock to the dog's paw.” 


Although not commonly cited in this connection, the Finch-Culler ex- 


periment thus clearly argues agal 
pretation of active avoidance learnt 
ras an inte 


nst the classical conditioning inter- 
ie and constitutes vet another line 
rvening variable. The authors are, 


of evidence in favor of fe 
nking of these 
If we think of fear as the conditioned. reaction 
and of paw lifting as an instrumental habit ot 


then shock to the chest could serve as an 


of course, in error in thi experiments as showing higher- 
order conditioning. 
In these experiments ( 
adjustment made thereto). 
unconditioned or reinforcin 
the leg or paw of the dog. 


e stimulus for fear quite as well as shock to 


\Mowrer-Lamoreaux experiment suggests (p. 73). one could 


m 
Actually, as the 
asive leg-retra 


probably even start the defer on response with the thoracic shock. 
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Eccher & Culler next report: 


Broaden (1939). instead of punishing the animals upon failure to me 
in eee ssively higher orders |as Finch & Culler had done, by means o 
thoracic shock |, rewarded them with food when they did respond. He was 
thus able to extend conditioning as far as the fourth-order (p. 223). 


Here again there is a question of interpretation. It is certainly truc 
that onc cannot legitimately think of food as an unconditioned stimulus 
for paw retraction, but neither is it legitimate to speak of the results 
which Brogden obtained as representing higher-order conditioning. As 
is now well known, food can readily and directly impart secondary rc- 
inforcing power to any stimulus associated therewith; and this is all that 
is needed to keep alive a bit of behavior such as leg retraction, This 


statement follows in part from considerations advanced in this chapter; 
but will be more completely amplified in Chapter 7. 
Then Eccher & Culler say: 


In the course of further experimentation with higher-order motor con- 
ditioning in cats, we noted quite by accident and much to our surprise, that 
the first order CR (bell-flexion) showed no sign of impairment when used 
as the conditioning agent in second-order (tone + bell). The bell-response 
alone, when not supported by shock reinforcement, quickly extinguished 
in the usual way; but when used for second-order conditioning, it became if 
anything increasingly vigorous and stable Ue 223). 


The unusual curves shown in Fig. 4-7 record systematic evidence on 
this score. The curve extending from o to 10 along the baseline depicts 
the incidence of paw retraction in 24 cats in response to a bell which, if 
not responded to, was reinforced by electric shock to the paw. The 
curve which then extends downward shows the progressive extinction 
of this response (in two animals) when the bell was presented and wot 
followed by shock if there was no leg flexion. The continuously ascend- 
ing curve, on the other hand, shows what happened when i1 animals 
were subjected to higher-order conditioning 


g, involving tone and bell 
(but no shock). And the final descending curve (for one animal) 


shows the extinction of the paw-retraction response to the second- 
order conditioned stimulus (tone) when this stimulus was no 
reinforced by the first-order conditioned stimulus (bell).™ 
Here it does indeed appear, as Eccher & Culler 
first-order conditioned response (bell-response ) i 


longer 


state, that when a 
s "used for second- 


" Phe animals not. otherwise accounted for in this experiment (from the 24 
which received. initial training) were used for other purposes which are of no 
immediate interest. 


TWO CONCEPTIONS OF SECONDARY REINFORCEMENT 121 


it becomes if anything increasingly vigorous and 


order conditioning, 
interesting features of this experi- 


stable.” But there were some very 
ment which might easily escape one's attention. It should be noted, 
first of all, that in the second stage of this experiment, wherein 11 


animals received a combination of tone and bell, the bell followed the 


tone oil y on those trials when there was no response to the tone. ("The 
reader is reminded that in group III the bell is sounded only when 
subject fails to react to the tone,” p. 228.) Se the procedure used here, 
ng, involved intermittent rather than 


in this second-order conditioning 
id differently, the conditioning pro- 


al. This procedure, it scems, 


continuous reinforcement; or, 
cedure was instrumental rather than cla 
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Successive tenths of total time 


om Fecher & Culler (1941) showing the results 
d two forms of extinction. In the curve extending 
as paired (instrumentally). with shock to 


Fig. 4-7. Curves reproduced fr 
of two forms of conditioning an 


from o to 10 (along the bas line), bell w P s; E 
cats forepaw; and when shock was discontinued, leg flexion to bell alone extin- 


guished as shown. But during the intermediate third of the experiment, tone and 
bell were paired instrumentally (in second-order conditioning), with tho Peale 
shown, When bell was discontinued, leg flexion to tone also extinguished. The 
ong the baseline r 
called Vincent curves. 


esult from the combining of data from 


somewhat unusual units 
several animals into so- 
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had two distinct advantages: (a) it meant that the reinforcing potential 
Vac E Is 
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of the bell was "used up" less rapidly than it would have been in a 
classical-reinforcement procedure; and a thc defensive reaction of 
jaw-lifting was also learned more readily than it would have been if 
bell had followed tone willy-nilly. 


Eccher & Culler put it this way: 


The CR to bell, which is now used as the conditioning agent, becomes both 
more brisk and more vigorous; hurried instead of deliberate. The cat gives 
every appearance of “fearing” the bell even more than the shock. Instead of 
delaving as in first-order it now scems anxious to get the foot out of the 
danger-zone with utmost dispatch. We hesitate to introspect for the cat or 
to speculate upon its experience; but the increased force of the CR is un- 
deniable (p. 230). 


Although it was Brogden, Lipman, & Culler who, in 1938, reported 
the experiment with guinea pigs in the revolvable cage which did so 
much toward discrediting classical conditioning theory, at least as an 
explanation of active avoidance learning, and so strongly suggested 
the concept of fear as an essential intervening variable, vet it is apparent 
that in their 1941 paper Eccher & Culler were still hesitant and apolo- 
getic in their use of the fear concept. As we have seen near the beginning 
of this section, they would have been on much sounder grounds if they 


had been more daring. What they needed to recognize was that a 
stimulus. for example the bell, which precedes shock—any shock, 
tends to acquire the capacity to arouse fear and that the reduction of 
this fear can then be used to reinforce ¢ 


ay given item of behavior, for 
example, leg flexion. And fear, of course, has aversive properties in its 
own right, so that once acquired as a regular response to bell, an organ- 
ism will do what it can to prevent its occurrence. Thus, if tonc precedes 
bell. it is not surprising if some fcar gets conditioned to it; but this fear 
will be less strong than that which occurs to thc bell. so there is a nct 
gain if the animal responds to the tone rather than allow ing the bell to 
occur (just as there is a gain, in first-order conditioning, in responding 
to the bell rather than allow ing the shock to occur). 

But what is most immediately pertinent about. the experiment. de- 
scribed by Fecher & Culler is their introduction of a procedure which 
is a sort of forerunner or counterpart of the methodology developed 
and employed by Zimmerman to such dramatic advantage (Section V ). 
Skinner. in the experiment described in Section IV. found that if he 
associated the click of the food-delivery mechanism 60 times with the 
delivery of a small pellet of food, this click alone would serve to rein- 
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force rats for 30 to 60 bar-pressing responses (without food). Now 
What Zimmerman did, in effect, was to repeat the Skinner experiment 
but to use intermittent reinforcement both in the training (click, food) 


and the test ( bar-press. click) phases of the experiment. In other words 
an intermittent classical con- 


in the first part of the experiment he used 
ad in the second part he used an intermittent in- 


& Culler did was to use two instrumental 
in succession, the onc involving first- 
order, the other second-order, conditioning. But the very fact of in- 
strumentality meant that the procedure also involved intermittency; 
in the Fecher-Culler experiment, 
intermittent. reinforcement—but 
including the fact that the Eccher- 
litioning of fear (or secondary 


ditionine procedure, ar 
strumental, What Fecher 
avoidance training procedures 


sO. thar one may say that here, TOO, 
SNAG! ut procedure involving double 
With certain differences, prominently 
Culler experiment involved. the conc 
motivation), whereas the Zimmerman experiment involved the condi- 
reinforcement). In both instances re- 


1 was observed, the fuller understand- 
oned to Chapters 11 and 12. But they 
can have considerable potency 


tioning of hope (or secondary 
Markable resistance to extinctior 
ing of which will have to be postp 
indicate clearly that both fear and h 
„also great persistence. 

in the next section, to take our conceptual 
we are and where we are going. 


ope 


and, if properly handled 
Now it will be useful, 


bearings and see, more precisely, where 


1 + 
VII. The Argument Summarized ' 
rs we have seen that, by the turn of the century, 
advanced two quite different. (though 
of the learning process, the one 
bstitution and the other stressing 


In previous chapte 
Pavlov and Thorndike had 
equally behavioristic ) conceptions 
Is su 
ase substitution. Hull attempted, un- 


and many psycholo- 


stressing conditioning or stimul 
or respor 


trial-and-error learning i 
these two theorie 


Successfully, to “integrate” 


"One of the learning-theory experts who read and criticized the original 
Manuscript of this book found the use of these recurrent, cummulative summaries 


tedious. Others may have a similar reaction. om 
1 s apparent that students profit from and are grateful 


he pro. ession 
an unchanging refrain or “chorus.” after cach new 


However, from extended experience 


m teaching this material, it i 
for this arrangement. Even € 
martes to be sheer repetition of 


Verse". (chapter). Instead, thc 
r-all argument 


al psychologist will not find these sum- 


summaries attempt to remind the reader of the 
and to show how the just preceding dis- 


general context or ove zd 
cussion logically fits into ir. If. a reader finds the summaries genuinely 


nonfunctional, he can, of course. 


however, 
easily omit them. 
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s continued to assume that Pavlov and Thorndike had identified 


gist s E TRI E 
5 distinctly. different forms of learning. This is what may be called 
) = - 7 5 


x first two-factor, or two-process, conception of learning. 

But then it gradually became apparent that punishment. does not 
"stamp out" stimulus-response bonds (or habits) in the direct way 
which Thorndike had supposed and that so-called defense or avoidance 
behavior is likewise not directly conditioned, as Pavlovian theory im- 
plied. In the latter connection, fear was soon introduced as an es- 
sential intervening variable; and the same phenomenon was used to 
derive a new (conflict) theory of punishment. 

On the basis of these developments, two-factor theory was then 
modified in the following way (to become version two). Condition- 
ing continued to be accepted as a real phenomenon, but it was re- 
defined so as to apply only to the acquisition of new emotional 
reactions, of which fear was taken as the prime example; and the 
acquisition of fear was assumed, more specifically, 
upon the conjunction of a formerly neutr; 
of some noxious stimulus or drive (such as electric shock). By con- 
trast, the termination or reduction of such a drive was assumed to 
provide the conditions necessary for habit form 
inforcement type-1). Emotional 
identified as 


to be dependent 
al stimulus with the ovset 


ation (and secondary re- 
learning (conditioning) was thus 
igi learning and habit formation (trial and error) as 
solution learning, In the acquisition of active and passive avoidance 
behavior, it was assumed that both forms of learning are involved: fear 
becomes conditioned to some formerly neutral (independent or rce- 
sponse-dependent) stimulus and the organism develops, as a “habit.” 
whatever response will eliminate the danger signal and reduce the 
attendant fear. Thus, whereas sign learning tuns restricted to emo- 
tional conditioning, solution learning Was extended to include learn- 
ing wher secondary, as well as primary, drives 

This revised version of two-factor | 
advantages and strengths, but it also had two major weaknesses: it 
continued to accept, essentially unmodified, "horndike's conception 
of habit ("bond") formation: ; 


; and it did not ad 
explain what has been referred to in this ch 


are concerned, 
learning theory had numerous 


equately recognize or 
; à apter as secondary rein- 
orcement type-2 Here we have seen that a stimulus with type-2 
secondary reinforcement capacity can exhibit this capacity in cither of 
two, quite different Ways: (a) it can produce higher-order condition- 
ing or (b) it can set up a new "habit" s 
the tvpe-2 secondary reinforcer 
which is to be associatively shiftec 


of some sort. In the one case. 
precedes and produces the response 
l (conditioned) to the new stimulus; 
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Whereas in the other case it follows the new response which is to be 
selectively) strengthened. Thus, one and the same stimulus, if it has 


tvpe-2 reinforcement capacity, can serve these two distinctive func- 


tions. Pavlov, as we have seen, was especially interested in the first, 
Hull in the second." 

Although both of the effects of type-2 secondary reinforcement 
have been repeatedly reported in the experimental literature, the evi- 
dence suggested that these phenomena were, at best, transitory and un- 
stable; and some investigators even questioned their reality altogether. 
In the present chapter new and dramatic proof of the existence and 
potency of secondary reinforcement type-2 is reported, so we are 
justified (in the following chapters) in making a careful and scarch- 
ing analysis of this phenomenon from a more theoretical and interpre- 
' 1 we shall find that tvpe-2 secondary rein- 
1 to bring about a change in our thinking 
al and as important as the change which 
in our thinking about 


tative standpoint. In the enc 
forcement is probably destinec 
about habit formation as radic 
secondary reinforcement. tv pez! produced 
^unishment and active avoidance learning. 


"Ns implied carlier in this chapter (Section IV), the distinction between these 


two functions of a secondary reinforcer rst 
i analysis of the Zimmerman (or any similar) ex- 


is used to establish a new habit, it actually 
ler conditioning. If we assume that the 
of some sort (see Chapter 5), 


(S') is not so great as it at first appears 


to be. For example, a careful re- 
eriment will reveal thar when an S 
achieves this etfect by means of higher-ord H 
secondary reinforcer itself. produces a "response 3 hich we wisl die 
and if we further assume thar any given bit of behavior X Wigs we wish to make 
ar pressing) regularly produces a distinctive pat- 
tern of sensory feedback as it occurs, ie response-correlared stimuli, then the 
à : "unconditioned" stimulus for getting its "response Cof 
d, or conditioned, to the stimuli which the to- 
This, as already remarked, is higher-order condi- 
ygency of this way of thinking will not be fully 


abitual (such, for example, as b 


S' in etfect, serves as the 
Secondary reinforcement) shifte 
-habituated action produces. 


tioning pure and simple. But the c ie i a 
ig, E d 1 i ch is emerging 

apparent until the new conception of habit w = i footnote herci. i 

roceeds becomes fully explicit. in Chapter 7. Uns Bayar a a 

sense. Qut f k r; bur it will help prepare the reader for this: new ibnetgnón 
» Out of order, 


Of habit when it is later presented. 


as our discussion 


Sccondary Reinforcement: 


A Unifying Theory 


In light of the very substantial evidence reviewed in previous 
chapters, there can be no reasonable doubt as to the reality of either 
type-1 or type-2 secondary reinforcement. The question is now no 
longer one of fact but of interpretation, explanation, theory. When 
behavior terminates a danger signal (cither an independent or a re- 
sponse-correlated one) and, as a result, becomes increasingly likely to 
occur in thc presence thereof, we are justified in inferring that the 
reinforcing agent is the fear reduction which presumably. occurs when 
the danger signal ends. An occasional attempt is still made to explain 
such learning in terms of classical conditioning or in some other ultra- 
objective way; but the evidence clearly favors the notion that sec- 
ondarv reinforcement, of the tvpe-1 variety, is here involved. By 
contrast, although secondary reinforcement type-2 is now also firmly 
established as an empircial fact, there is little or no agreement as to 
how it is mediated. The purpose of the present chapter is to put 
forward, and cite evidence for, a theory. of type-2 secondary rein- 
forcement which will bring this phenomenon into a consistent and 
systematic relationship to secondary reinforcement type-i. 


I. Type-2 Secondary Reinforcement as 
Decremental Fear Conditioning or "Hope" 


Perhaps the best w av of beginning this discussion will be to revert 


to the conceptual scheme suggested in Chapter 2 to account for fear 
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conditioning, of the incremental variety. For convenience of reference, 
Vig. 2-1 is reproduced here as Fig. 5-1. This diagram involves an 
assumption which invites further consideration and suggests some in- 
teresting possibilities. The present writer had originally taken the 

n reaction" 


position. that. fear is a "conditioned. form of the p: 
CMowrer, 1939; see also Chapter 4. p- 93). If, for example, electric 
shock is thought of as an unconditioned stimulus and pain as the 
unconditioned reaction, then the fear reaction elicited by a formerly 
neutral stimulus, which has been paired with the shock, would be the 
conditioned reaction. But N. E- Miller (1948b) was probably closer 


to the truth when he suggested that fear is instead a part of the total 
response innately produced by a painfully intense stimulus, such as 
electric shock, and that, in conditioning. this particular response com- 

hich is not conditionable, or 


ponent (unlike the pain component, W 
only weakly so) gets shifted from the UnCS to the CS, in the manner 


Of classical Pavlovian theory." 


Fig. 5-1. Commonly used concep- 
tual scheme for indicating the cir- Cl oce a | 


cumstances under which the emo- signal rer 


an b 


Fear 


tion of fear, initially aroused by 
“unconditioned” stimulus like elec- 
tric shock, becomes conditioned to 

a formerly neutral stimulus which Shook o Se 


has been paired with onset of shock. 

at fear occurs to shock, along 
ling pain). but that it alone is 
iously suggested was that 


In other words, Miller argued th 


With various other reactions (incluc 
as the view prev 
and that when it becomes 
need as fear. As far as 


readily conditionable; where 
pain is the principal reaction to 


Conditioned, it is transformed into and experic ! 
FOR tas miconditioned. OF response) is concerned, these 


shock 


anticipatory. 
re concept advanced by Hull in his 
If the total reaction to electric 
then fear, as a “fr reof, may 

rar will be recalled, stands for ; and one 


be denoted as re. However. the “g 
speak of fear and the other responses aroused by a 


oal reaction." 
nconipass both type-t and type-2 secondary 


acks the desired generality. A preferable 


ET OR : ; y 
Miller's notion here is analogous to the 


discussion of type- secondary reinforcement. HF d 
: tional component” the 


Shock 3 
dock is represented as Re, i 
goal 


c: DNE 
annot very meaningfully 3 : 
x E Because we are here attempting to 


Puntul electric shock asa £ 
develop a way of thinking that will e 
reinforcement, Hull's re notation. thus l 


Notation would be Ry, to refer to the total reaction | 
forcement, and re to indicate the conditionable (fear). component thereof. In 


like manner, Ra nore the rotal reaction to decremental reinforce- 


Ment (reward) and rm as the 


on elicited by incremental rein- 


in be used to de 
conditionable 


(hope) component. (sce below). 
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two ways of thinking come to much the same thing. But they have 
quite different implications when one turns to a consideration of the 
reactions or states of mind which are aroused by the noxious stimulus 
itself. It is with the latter that we are now especially concerned; and 
here we find that Miller's formulation lends itself quite ideally to our 
purposes. 


Fear Fig. 5-2. Schematic indication of the probable 
x relationship between termination of a noxious 


Shock 


stimulus, such as electric shock, and the ac- 
companying emotion of fear. 


If, as Miller suggests, fear is aroused by the onset of a noxious (un- 
conditioned ) stimulus, we may reasonably infer that fear continues 
to be experienced (in some degree) as long as this stimulus is present. 
The question which then arises is: What happens to the fear when 
the unconditioned stimulus terminates? Since studies involving. the 
galvanic-skin-reaction technique show that, following shock onset, the 
fear reaction builds up rather slowly (as indicated in Vig. iu oS 
reasonable to suppose that it subsides, following the end of shock, 
roughly in the manner shown in Vig. . If this be true, then the next 
question to suggest itsclf is whether fear reduction, following shock 
reduction, can be conditioned to an antecedent. stimulus, just as fear 
induction following shock onset can be. If so, we are justified in show- 
ing, in Fig. 5-3. at the right, what is essentially the mirror 
counterpart, of Fig. 


image, Or 


1 reproduced here at the left. 


Danger | es Safety [———31 


signal ey signal 


z 
/ / Y \ 
Fear - Ses 
EM 


Fig. 5-3. Composite of diagrams shown in Figs. 
tures. As will be seen, a 


5-1 and 5-2, with two added fea- 
safety signal” has been 
right; and the broken lines suggest how the 
end of shock may become anticipatory, 


added to the diagram at the 
drop in fear that occurs after the 


If this schematization is valid, then we may assume (a) that sec- 
ondary reinforcements types ı and 2 both involve fear reduetion but 
that (b) the fear which is reduced in the two instances is at least 
Operationally different: in secondary reinforcement Uy pe-i, it is aroused 
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by and terminates with a mere danger signal; whereas, in secondary 
reinforcement. type-2, fear is aroused by some inherently noxious 
stimulus—which is “already present" —and is reduced by the appear- 
ance of what is in effect a safety signal, indicating that the noxious 
stimulus will soon end. Thus, we may say that in secondary reinforce- 
ment type-2 the subject will learn whatever behavior gets the safety 
signal; whereas in secondary reinforcement. type-1 the subject will 
learn whatever behavior gets rid of the danger signal. Superficiallv, 
l. getting and. getting rid 
al is dv nami- 


these two learning situations are antithetic 
of a signal of some kind. But getting rid of a danger sig I 
cally ( hedonistically) equivalent to getting a safety signal; and this is 
Why we may now “speak of these two forms of secondary reinforce- 


ment as being much the same in principle. 
The reader must. of course, bear in mind that the ways in which a 


danger signal and a safety signal, as here defined, acquire their sean- 
ings are indeed antithetic: sociation with oet 
of some noxious stimulus; 


a danger signal, by 
and a safety signal, by association. with 
the offset of such a stimulus. And it is precisely because of this dif- 
ference in meaning that the appearance of one type of signal is func- 
tionally equivalent to the disappearance of the other. As we shall later 
see, the "integrative," or “resolving,” power of this formulation is con- 
siderable; 2 but it involves a logical complication. which was first 
pointed out to the writer by Mr. Murray S. Miron. 

r 3. he will be reminded that 


If the reader will again refer to Fig. lc 
Miller's notion that fear is an “unconditioned’ 


ulus, such as can be provided by 


We are here accepting I 
response to a painfully intense stimu! A : 
electric. shock, and that if such a stimulus is preceded by a signal, 
fear may become attached to it as a conditioned response. And there 


is, of course, nothing enigmatic about the hypothesis that if fear is 


aroused, as a conditioned response to a signal, it can also be allaved 
by termination of this signal. If the signal causes the fear, its termina- 


tion should also be a sufficien ie jt 
vond this point, we encounter a complication. In 


t condition for cessation of the fear. 


But in going be 


“More or less independently, a number of writers seem to be moving toward 
For example, Jourard (1958) writes as follows: “Ir 


this general point of view. ] b m s 
alid to think of the liking-response as a conditioned 


Is convenient, and perhaps v 


isely, as t € 
liking-response has, in the past, produced sat 


produced satisfaction. The liking-response is 


he affective component of an expectancy. The 


response, or more pre 
Object which induces the 


Or is related to something which E Sa : | c 
set for something satisfying to occur, just as anxiety is a 


(p. 195). Cf. also Henle (1956), 


ction, 


a sort of preparatory i S 
Preparatory set for something painful to occur 
MeGuigan (1956), and Olds (1954. P- 2327- 


23 
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the right half of Fig. 5-3 we are assuming that, if a signal precedes 
the end of shock, the reduction in fear which follows shock termination 
can also be conditioned to that signal and will, on later occasions, take 
place before shock termination. The crux of the dilemma posed by 
Mr. Miron, then, is this: How can fear be an unconditioned response 
to shock and yet be capable of anticipatory (learned, conditioned) re- 
duction, on the basis of the occurrence of a mere signal of shock 
termination? Or, in different words, if shock is a sufficient and neces- 
sary cause of fear, how can the addition of another stimulus climinate 
the fear, while the shock is still present? 

This is obviously an important question. There are, however, two 
plausible and mutually compatible answers. 

For a number of years it has been known that a primary drive, 
such as an electric shock, may have, in addition to drive properties, 
also signal or cue properties. This can be readily demonstrated in the 
following wav. Suppose that a “naive” (i.c. previously unshocked) 
rat 1s put on ag 


ill and that an electrical charge is gradually imposed 
on the grill, starting at zcro intensity and building up. progressively, 
to a relatively high intensity. Suppose, also, that after reaching this 
intensity, the shock is suddenly turned off and that after a rest interval 
of two or three minutes, the shock is again gradually. built up. On the 
second or, at most, third presentation of shock in this way, it will be 
observed that the rat starts "churning" its fect, running, jumping, 
and squealing at a considerably lower intensity of shock than w 
necessary to produce these reactions on the first trial, 


as 


How is one to explain such an observation? It does not scem likely 
that the animal has become literally “more sensitive" to shock, so 
that a weaker intensity “hurts” just as much as a stronger intensity 
originally did. The usual effect of repeated exposure to and experi- 
ence with electric shock, short of actually damaging intensities, is to 
make one somewhat "adapted" to it, More likely 
under the circumstances described, the shock, 
becomes a "conditioned stimulus," 
of shock hich are to follow. 
shock, as a primary drive, ¢ 


is the possibility that, 
at the weaker intensities, 
or signal, for the stronger intensities 
This would mcan that, though the 
loes not become subjectively any stronger 
at the low intensities, these intensities of shock cue of} relatively 
strong fear, which in turn helps to motivate, now, the 
various forms of agitated behavior which are even- 
tually produced by shock alone." 


more quic 
. on the first tr 


"Observation of a parallel. effect, 


with hunger drive, has been reported. by 
Mandler (1958). Here rats which had 


previously, and repeatedly, been subjected 
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Taking this demonstration as a paradigm, we may therefore reason 
thar, especially with drives which gradually mount in intensity (as 
primary drives often do) but even with drives of relatively sudden 
onset and continuous intensity, there is a tendency for fear to occur 
in response thereto, both on a conditioned and an unconditioned basis. 
Hence. to return to the Miron paradox, we conjecture that it would 
be possible for at least the conditioned portion of the fear produced 
as electric shock to undergo anticipatory reduction 


by a drive such : 
termination, The uncondi- 


in response to a signal of impending shock i | 
tioned portion would, of course, by definition persist until the shock 
itself terminated; bur since it appears that the conditioned portion of 
the total fear elicited by a primary drive (or “unconditional Y stimulus) 
may be considerable, there is still ample latitude for secondary rein- 


forcement, as fear reduction in advance of primary drive reduction, 


to operate in the manner hypothesized. 


ll. The Miron Paradox Further Considered 


been said, in the preceding section, is clear and 


If what has thus far nee 
rhaps go at once to Section Ill. 


convincing to the reader, 
If not, he may profit from the 


One of the things that may j d "E : 
obseure is the facr thar in che example given, of a grac ually mounting 


both the conditioned and the unconditioned stimuli for 
of electric shock) impinge upon the 
modality. Ordinarily, in fear con- 
ditioning, we produce the fear, an unconditioned basis, hy a pain- 
t with it, for example, a light or sound of 
there is nothing different in using 
as a conditioned stimulus and a 


he should pc 
following additional discussion. 


make the foregoing analysis somewhat 


electric shock 
fear (ic. different intensities 
Organism through the same sensory 

on 


fully intense shock and patr 
some sort, However, in principle, | 
a faint shock. (which just “tickles” ) pom eS 
More intense shock as the Ens. Perhaps all this i5 € E: ki if we use, 
Not à continuous, gradually increasing shock, but two ¢ iscrete intensi- 
HS. [ -4. Here the weak shock is completely 
"naive," has no tendency to arouse 


ties of shock, as shown in Fig. 
innocuous and, if the subject 
fear. If, however, this intensity 


is \ 
of shock is followed a few times by 


reacted to muld deprivation by eating more than 
experimental Ss exhibited accelerated 
-Gnsummataty .. Within a Hullian framework 
i» efinition of drive (Do in terms of hours of depriva- 
ation experience of the animal must be 


later 
xperiments the 
behavior. - 


to severe food deprivation 
did control animals. "In both € 


and more frequent 
he d j 
Previous depriv 
b 


this finding suggests that t 
ton alone is inadequate: 
taken into account? (pp. 5167517 


132 LEARNING THEORY AND BEHAVIOR 


che higher intensity shown, it would be very surprising if the subject 
i rt reacting to the weak shock with fear, in anticipation 


did not soon sta r l ; i 
of the strong shock to follow. To the best of the writer's knowledge, 


no one has performed an experiment. of precisely this. kind, with 
either mice or men; but the outcome is pretty much a foregone con- 
clusion, especially since it has been observed, as previously noted, 
that with recurrent presentation of a gradually mounting shock of 
the kind described, evidences of fear occur sooner and sooner. In 
principle, there is no difference between the two situations. 


Fig. 5-4. Suggested scheme for making a faint 
electric shock serve as the conditioned stimu- 
lus, and a much higher intensity of the same 
stimulus serve as the unconditioned stimulus, 


in fear conditioning. 
Shock | CS i UnCS 5 


Or, let us consider a different, though not incompatible, solution to 
the Miron dilemma. If the idea of the fear, reflexly aroused by strong 
shock, being conditioned to a weak intensity of the same ‘stimulus 
which precedes it does not seem very attractive, we may think about 
the problem in the following w ay. If a rat experiences protracted 
shock in a given experimental situation, that situation. will become 
"fear-charged"; which is to say, the fear aroused by the shock will 
become conditioned to the preceding and accompanying constellation 


of stimuli which we call the "situation." Therefore, when our subject 
is put into this situation, he will experience considerable fear, in ad- 
vance of the shock being turned on; and when the shock is applied, 
the animal, having previously suffered protracted shock here, will 
again “expect the worst.” In other words, to the unconditioned pain 
and fear produced by the shock is now added the conditioned fear 
produced by the situation. 

If, now, after such training, the rat were given, let us sav, a tone for 
two or three seconds just before the shock terminated and the animal 
got out of the apparatus, the tone could easily acquire secondary re- 
inforcing capacity in the 


i - following sense. When the shock is on, the 
situation-without-tone is c 


luigerous, i the situation is one in which 
shock may continue for an unpredictable period of time. On the 
other hand, the situation-zzitb-tone (even though shock is still present) 


: Just suggested, because the subject will 
soon be delivered from shock am 


is not. dangerous, in the sense 


7 situation, Therefore the tone will 
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be heard with a feeling of "hope; w hich is what we mean here by 
“secondary reinforcement.” 

At first it may appear that this resolution of the Miron paradox 
is no different from the one first proposed. But the assumption here 
is that the fear which is capable of being "turned off" by the s fety 
signal is not (necessarily ) conditioned to the shock stimulus itself, 
but to the situation in which th 
sional occurs, it changes the situation, from one in 
" ed indefinitely to one in which it “will 
be over in a moment," Hence we do not here have to assumc cither 
that a conditioned. stimulus relaxes an unconditioned response (the 
Miron dilemma) or that it relaxes a component of fear that has be- 


come conditioned to the unconditioned stimulus. We may assume 
situation-with-shock-but- 


instead, simply, that there are fzzo situations: | i 
no-tone and situation-with-shock-and-tone, the one being quite 
ominous, the other much less so. And the specific stimulus, the tone, 
which makes the difference may be thought of equally as a cue, for 
making a “discrimination” between the two situations (sec also Chap- 
ter 12), and as a secondary reinforcer. 
As already noted, the nice thing about the two possible resolutions 
of the Miron dilemma advanced above is that they are not in any sense 
Either could be correct (or incorrect) without 
of the other; and, what seems most 


at stimulus has been experienced. And 


when the safety 
which shock may be experienc 


Opposed to one another. d 
necessarily prejudicing the validity d UD ME 
likely, they may both be valid and operate simultancously’. Thus we 
) x Trom what at first seemed like a formidable difficulty 
rive some new Ways of thinking which will stand 
us in good stead at several later stages in our analys s 

By way, then, of summarizing the discussion 1n this and the pre- 
ceding section, let us suppose that a shock of moderate intensity is 
G orill on which a rat is sitting. Assuming the 
rat to be naive with respect to such stimulation, the shock, wil be 
immediately felt as “painful” and the rat will show surprise and 
mildly: agitated behavior; and if the shock continues, with nothing 
o we see the rat gradually become more and more 
and unconditioned fear caused by the shock 
added a conditioned fear, i.e.. fear of more 


happily escape fr 
and, in so doing, de 


suddenly applied to the 


to be done about it. 
"worried." To the pain 
atany given moment is NOW 


pain. (One can conjecture that there would even be, not just a fear 


also fear of more fear, ies the fear to follow. This, 


of more pain, but 1 
and, later, F. D. Roosevelt meant 


presumably, is what William James 
by “fear of fear") Thus. although the p 
otal discomfort presumably mounts—and 


ain caused by the shock re- 


mains about the same, the f 
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may eventually result in a sort of "frenzy" or “panic.” Fskimos, it is 
said, in the old days sometimes committed suicide because of a pro- 
tracted toothache. It is not that the ache at any given moment is so 
unbearable, but rather that the fear of continued pain (and continued 
fear? ), cued off by the present pain (and fear? ), becomes intolerable. 
It is then, presumably, this conditioned clement in the total experience 
of discomfort that can be lessened by “reassurance” in human beings. 
or by a "safety signal" in rats, and provides the basis for hope. 


Ill. Empirical Evidence for Proposed Theory 
of Secondary Reinforcement Type-2 


More than a decade ago Mowrer & Vick called attention to a 
commonplace example of tvpe-2 secondary reinforcement, as follows: 


One of the commonest vet most dramatic illustrations of this phenomenon 
is the relief experienced when a physician is consulted. One is ill and sutfer- 
ing from pain and inconvenience. The phy sician arrives, diagnoses the diffi- 
culty, prescribes treatment, and intimates that ina day or two one will be 
quite hale again. It is unlikely that the examination or the insuing exchange 
of words has altered the physical condition of the patient in the least; yet he 
is likely to “feel a lot better" as a result of the doctor's call. What obv iously 
happens in such instances is that initially the. patient's phy sical suffering is 
complicated by concern lest his suffering continue indefinitely: or perhaps 
grow worse. After a reassuring diagnosis, this concern abates; and if, subse- 
quently, the same ailment recurs, one can predict that it will arouse less ap- 
prehension than it did originally (provided, of course, that the physician's 
reassurances were valid and his treatment effective) (Mowrer & Vick, 1948, 
P- 193). 


Or one might consider a group of desert travelers who, almost dead 
from thirst, catch a glimpse of an oasis in the distance. Immediately 
they will feel relieved, hopeful. Their thirst will not as vet have les- 


"A thoughtful student, with considerable background in Gestalt psychology 
(especially as taught in schools of education) has remarked upon the similarity 
between the foregoing ideas and the Gestalt conception of closure, in the sense 
of "completion." “This term is used, particularly in the area of perception, to 
indicate the act of mentally completing a. physically incomplete figure so as to 
give it a "good gestalt, familiar form, a definite meaning. But it seems that 
the term is also used, more informally, to imply completion in the sense of con- 
summation of a need, the satisfaction or reduction of a drive. Thus, a type-2 sec- 
ondary reinforcer may be said to give promise of "closure" or drive reduction; 
and the reduction of drive thus tor 


ast is the closure. This interpretation is un- 
objectionable so long as one remembers that the secondary reinforcer, in pre- 
dicting primary drive reduction (closure), itself mediates a form of drive reduc- 
tion, namely secondary drive reduction. (Sec also Chapter 9.) 


SECONDARY REINFORCEMENT: A UNIFYING THEORY 135 


will be reduced. The oasis is 
ld undoubtedly be powerfully 


sened, bur the fear engendered thereby 


a "salety signal.” and its appearance v ou 
reinforcing, albeit only secondarily so. 

; Intuitively compelling as they may be, suc 
tions are nor, however. the stuff. of which an exact science is made. 


Vherefore, it is fortunate that evidence of a 
available. In his review of the evidence for secondary reinforcement 


already: alluded to, Miller (195 ib) writes: 

-addicted chimpanzees will strive to 
rned to associate with an in- 
reinforced by a reduction in 
eriment it seems probable 


h commonplace illustra- 


more rigorous kind is 


Spragg (1940) show ed that morphine 
pee the syringe and other objects they have le: 
jection, Sometimes the use of morphine may be 


fear like that described for alcohol, but in this exp i 
that the motivation was produced by phy siological withdrawal symptoms 


and hence was not a learned drive. Furthermore, the animals worked for the 
syringe only when they had withdrawal symptoms. Later in the experiment, 
5 ` ions suggested that temporary relief from with- 
luced by injections of physiological saline. 
oduct of learning and would be expected 
avolved in the injection (p. 462). 


however, incidental observat 
draw al symptoms could be proc 
Such relief must have been the pr 
to give reward value to the cues it 
an excellent paradigm for the conception 
re under discussion. It is to be inferred 
being severely painful, not 


Vhis observation provides 
of secondary reinforcement he 


thar the morphine withdrawal symptoms, 


f reflexive fear but also a good deal 
ar—fear based upon the possibility, 
c, that the present suffering might be followed 
worse suffering. An injection of morphine 
had, in the past. brought relief; so it is understandable, if the present 
line of thought is sound, that the mere prospect of such relief (at 


the physiological level) would be somewhat relieving (emotionally). 
t » c confirmed bv the observation that the 


on the first few times it was tried— 
blem arises) by an injection, not 


only aroused a certain amount o 
of conditioned or anticipatory fe 
born of past experienc 
by more and, quite possibly. 


Vhis supposition seems to b 


animals were calmed. (at. least 


and then the familiar extinction pr? 


of morphine, but of an inert salinc solution. 
an instance of the w cll-known placebo effect,” 


as manifested at the human level Sand an illustration of the general 
principle of suggestion. ‘This term has not been previously used in 
this book; and, to the W rite lge, it has not previously been 


discussed by others in the conte 
Bur, from the foregoing 


ACE 


the administration of 


Here, obviously. i5 


rs knowlec 
xt of contemporary learning theory. 


considerations, it will be clear that so-called 


S. Brown's (1955? comment: “Wolff and Goodell (1943) have found that 
1 placebo falsely described as aspirin. can produce an actual 
old. for pain” (p. 175) 


increase in a subjects thresl 
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suggestion (of at least one kind) is but another name for the ctfect 
produced by a type-2 secondary reinforcer: it arouses hope or. 
equivalently, relieves apprehension and in so doing makes the individual 
feel better without, in a more fundamental sense, really being "bet- 


ter.” This effect may be termed positive suggestion, to distinguish it 
from the opposite situation, wherein an individual is made to feel 
worse without, for the moment at any rate, really being any the worse 
off. In the latter c we speak of the arousal, not of hope, but of 
fear—to which, in like manner, we might apply the term, zegatice 
suggestion. 


More systematic evidence for the phenomenon here under discus- 
sion is reported by Flithorn, Pierey, & Crosskey (1955) in a paper 
entitled “Prefrontal Leucotomy and the Anticipation of Pain." "These 
investigators pose the problem thus: 


Since our own observations and those of others suggested that the frontal 
areas [of the human brain which are] damaged by leucotomy contain no 
primary vegetative centres, an attempt was made to devise a test which 
would measure autonomic changes which could be interpreted in terms of 
the psychological effects of leucotomy. A typical effect of a prefrontal de- 
structive lesion, whether therapeutic or accidental, is an apparent reduction 
in the patient's feelings of anxiety. Further, an carly paradoxical observation 
was that patients who suffered from severe pain, organic in origin, might be 
relieved of their suffering although their perception of pain remained un- 
impaired (Chapman, Rose, and Selomon, 1948). With a view therefore to 
analysing further the changed attitude to pain which follows prefrontal 
leucotomy the following test procedure was devised (p. 34). 


It is not necessary to give the details of this study; but the relevant 
facts are contained in the summary, which reads as follows: 


In order to test the hypothesis that prefrontal leucotomy relieves painful 
conditions by reducing the anticipatory clement of fear, a pain-expectancy 
test was devised, The relative disturbance caused by a painful shock and by 
a preceding warning light was estimated by measuring the ratio of the psv- 
chologalvanic responses aroused by these two stimuli. Twelve of the 13 
patients examined showed post-operatively an increase in this ratio, which 
indicates a relative reduction in the autonomic disturbance caused by the 
warning signal. It was shown that this reduction in the anticipatory fear as- 
sociated with a painful stimulus was not due to an alteration in the percep- 
tion of pain or to a reduction in the amount of pain tolerated during the test. 

One subject, who had been relieved by leucotomy of a totally incapaci- 
tating preoccupation with a post-herpetic neuralgia, is reported in. detail 
(PP 42743)- 


The fact that the painful electric shock continued. to produce a 
vigorous psvchogalvanie reflex indicates that the reflexive, or uncon- 
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ditioned, response of fear to pain is still intact after leucotomy; but 
the fact that a signal that shock is imminent produces, in Icucotomized 
individuals, a diminished 
authors note, that "this reduction in the anticipatory fe: 
specific effect of the operation. In other words, it appears that the 
relief. produced by leucotomy in cases of intractable pain is achieved 
) acity to "worry." i.c. to experience co/- 
i ng either reflexive fear or sensitiv- 


ssvchogalvanic response suggests, as the 
Py EI sso 
is a rather 


by reducing the patient's cap 
ditioned fear, rather than by reducir 
itv to the noxious stimulation (pain) itself. 

If, therefore, the element of discomfort ( 
tors in intractable pain is great cnough to justify 
asure as leucotomy, we are here dealing 


or “motivation” ) added by 


purely emotional fac 
so radical an interventive me 
with no purely hypothetical construct, invented solely for purposes 
of theory construction. Put another wi what leucotomy apparently 
in cases where pain itself. cannot. be controlled 
a result, has developed a state of chronic 
this operation destroys the capacity to 
anticipatory fear,” and thus spares the 
otherwise unavoidable suffering. 


accomplishes is this: 
and where the patient, as 
apprehension and hopelessne 
experience apprehension, or 


patient one important element of his 
in extremis, the mechanism whereby con- 


Here we presumably sec, 
antithesis, hope, normally operate. 

recently begun to explore the possibility 
involves fear reduction no less 
Writing in 1954, Farber 


ditioned fear and its ant 

Psychologists have only 
that secondary reinforcement type-2 
; reinforcement type? 


than. does secondary ; 
f reument as of that date, thus: 


summed up the status of this a 
problems have been met has been to suppose 
that these latter. events, usually designated be "secondary reinforce w do 
xot ener We reducit à drive: Mow rer, in discussing Estes? results, 
has speculated ihat s us when an organism 15 subjected to strong primary 
drive, there is commonly added to this the secondary motive of anticipated 
continuation of the prim: ry drive” (Mowt » 1950b, p: 149). lr is this drive, 
according to. Mowrer's view, that might be reduced by secondary rein- 
feurs [And] Whiting, in personal. communication, has suggested that 
STE rede EOI “insecurity might be regarded as an acquirable 
portion of drive states resulting from deprivation, just as fear or anxiety is 
regarded nskati acquirable portion of the drive state resulting from noxious 
stimulation (Dollard & Miller, 1950)- , , 

A somewhat different conjecture W ith respect to à possible drive-reducing 

* Further evidence for this view is provided by Hill, Belleville, & Wikler (1954) 
buen tax way, af OM experiment with rats, “The results are analogous to those 
of the earlier studies on human subjects and support the hypothesis that reduction 


of anxiety: associated with 
analgesic” (p. 183)- See also Barber (1959)- 


Another way in which these 


anticipation of pain is one necessary action of a potent 
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mechanism for secondary reinforcers has been discussed by Osgood (1953). 
Pointing out that there is evidence that specific hormones are associated 
with satiation as well as with hunger, he suggests that stimuli associated 
with drive reduction might serve as conditioned. stimuli, releasing the hor- 
mones that accompany satiation. 

It is probably not necessary to observe that these suggested mechanisms 
are hypotheses to be investigated, rather than conclusions drawn from em- 
pirical evidence. But I believe most psychologists would agree that the drive- 
reduction hypothesis has not been so unproductive that the investigation 
of these possibilities and their implications is unwarranted (pp. 7-8). 


And even Hull, who did so much to popularize the r, 
of secondary reinfor 


—s, conception 
ement as a form of stimulation rather. than 
drive or stimulation reduction (scc Chapter 4. also Chapter 6). in his 
1952 book, Essentials of Behavior, took a position with respect to sec- 
ondary reinforcement basically similar to the one presently under dis- 
cussion. Hull said: 


We must now observe that in a behavior sequence such as that of a rat 
sing from a compartment where its fect arc receiving electric shocks on 
Its way to a non-shock goal or compartment, there occurs a cessation not 
only of the shock stimuli but of the proprioceptive stimuli arising from the 
running. This major reduction in stimulation constitutes, of course, the con- 
crete substance of the | primary] reinforcement of the running or escape 
activity, quite as ordinarily aroused. But it also constitutes much more, and 
this is the main point to observe: the cessation of the shocks not only brings 
the locomotor escape [sic], but also a general relaxation of the autonomic 
activity called fear. This relaxation process appears to become attached, 
much as positive activity would, to the stimuli active at the time, together 
with the traces of carlier stimuli. As a result, on later repetitions of the ob- 
jective conditions in question this relaxation. generalizes forward on those 
traces, and gives rise to conditioned inhibition. This has appropriately been 
called by Miller (1951b), anticipatory relaxation (pp. 26-27). 


This is one of the very few occasions when Hull ever made for- 
mal reference to the concept of fear; and his infelicity in dealing 
with it here attests its unfamiliarity. to him. However, it would be 


unfair (after the treatment of Hull's theorizing in Chapter 4) not to 
make reference to this gesture on 


his part in the direction of another 
kind of thinking.? The pass: 


age quoted serves, also, the useful purpose 


* Although the passage just quoted ¢ 


an hardly be squared with the view that 
secondary reinforcement type-2 


involves stimulation iverease (as Hull elsewhere 
stipulated), one can, however, reconcile it with the notion of r 


of Re as the totality of responses which 
ticipatory relaxation" of fear is 


s If one thinks 
occur to shock termination, then "an- 
à component thereof, representable as re. But 
here one stops with re and does not extend it to include the notion of r—s, (ef. 
Chapter 8; also footnote 1 of this chapter). 
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of leading up to the problem w hich is to be discussed in the next sec- 
tion, First, however, it is instructive to note some observations rc- 


Richard D. Walk, of Cornell 


ported ( personal communication) by Dr. 
University, He savs: i 

Recently ] have been working on a paper tentatively titled, “Some pa- 
rameters of fear.” This paper is based on material collected at Fort Benning, 
the trainees, at the conclusion of their first 
“In your own words, describe 
i ` Their reports refer 
when the parachute 


Georgia, in 1953... + | asked 
o the following: 
achute jumping. 
a the sequence, 
1 for the plane, ete. 


five parachute jumps, to d 
some of vour fear reactions during pat 
to the feeling of fear at many points ir 
Was fitted on, the wait in the Sweat shed 

looking at the data one gets the impression of a period of anticipation 
Which is turned into intense fear at the command, “stand up, hook up." This 


Is analogous in many respects to the conditioned stimulus, in ordinary labora- 
ar diminish? All of the most feared events 


tory conditioning. \Vhere should fe: : s 
jumping from the door, the. parachute opening. the opening shock—oc- 
cur within 3 to 8 seconds; and many say thar relaxation comes, as might be 
expected, when the parachute opens However, the interesting thing is that 
Many report relaxation and lack of fear as soon as they stand in the door. 
This is not logical. None of the most feared events h ; been experienced and 
Yet the trainee relaxes. This puzzled me greatly until it occurred to me that 
the door had a special function—as 2 = for “hope of relief, hence relax 
tion. As a matter of fact, one tr nee even reported that he was ina high state 
of fie ag che qounumaitae ds and up.” and then he was unafraid. For him, 
hope? had moved forw ard even further in the sequence. 
in which anticipatory relaxation occurs 
ages We have been 


urs. In earlier p 
xious) stimulus is 


Here, then, is a situation 
before the traumatic event itself occurs arli 
discussing situations where the traumatic (painful, noxious) st! 
A E ie n sehr. However. in parachute jumping. the 
is the only really painful incident; and it is so 


shock of opening" 
the thought. Ir wi 


brief that apparently 
Intervene ip. advance 
the open door of the planc 
Apparently arouses. aS Dr. 
Which, on past jumps. has sl 
adds: 


ll soon be over now," can 


and produce a drop in fear. Because standing in 
is shortly followed by the jump. the door 
€ some of the relief reaction 
ved. Dr. Walk pertinently 


Walk suggests. 
hortly follov 


collected at the end of five parachute jumps, ] have 
the door was feared on the first jumps and ac- 
later ones, OF W hether it was a func- 
xperiments and some very practical 


Since my data were 
Hy Syd of knowing W herhe 
(uired fear-reducing proper 
tion of the individual. Some 


Implications are suggested by these 
ive discussion here, a fuller report of Dr. Walk’s 


available as 3 purple-duplicated monograph 


investieati se 
gation has become : oi ; 
entitled, “Fear and Courage: Á Psychological Report? (Walk, 1959).) 


ties on the 
interesting € 
findings. 


Coo late for extens 
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IV. Shock Termination and 
Secondary Reinforcement 


It would seem that electric shock should, in many respects, be quite 
ideal for use in the establishment and study of secondary reinforce- 
ment type-2, just as it has proved to be in the Investigation of sec- 
ondary reinforcement type-1. It can be turned on and off at will, at 
any desired intensity, and in other Ways is extremely convenient and 
effective. However, as regards the use of shock for the study of 
secondary reinforcement tvpe-2, the results have been highly incon- 
sistent. In an carlier draft of this chapter, a number of experiments 
which have been carried out in this connection (Barlow, 19 
pock, 1950; Smith & Buchanan, 1954; Nefzger, 19 
Brownstein, 1955; Montgomery & Galton, se 
Gleitman, 195 


Cop- 


; Goodson & 


À Solomon & Brush, 1956; 
5; Littman & Wade, 1955; Lee, 1951; and Mowrer & 
Aiken, 1954) were reviewed in detail; but this no longer scems neces- 
sary or desirable, for it is now evident that these investigations in- 
volved a number of misconceptions and artifacts which render them 
far from crucial. Moreover, in the meantime Beck (1957) has done 


a particularly thorough review of these studies and from them draws 
the following conclusions: 


1. Several contemporary behavior theories clearly predict that it should 
be possible to establish secondary reinforcement by pairing a neutral stimu- 
lus with the termination of shock. 

2. There is no clear-cut positive evidence in support of this prediction, 
and there are a number of experiments with negative results, 

3. The shock-termination experiments have not been ideally suited. for 
demonstrating [type-2] secondary reinforcement insofar as they have not 
met the conditions most adequate for demonstrating secondary reinforce- 
ment with other forms of primary reinforcement (food and water) gg 


One of the commonest reasons for failure to demonstrate that 
stimuli associated with shock reduction acquire secondary reinforce- 
ment capacity is this. In the study of this phenomenon in relation to 
other primary drives, such as hunger and thirst, it has been common 
practice to test the efficacy of the secondary-reinforcing stimulus 
with tbe primary drive still operative, f i 


‘Thus, if a given stimulus has 
been associated, 


for example, with hunger reduction, the secondary- 
reinforcing power of this stimulus will be tested while the subject is 
hungry. It is, after all, only reasonable that a promise of hunger rc- 


duction will be most effective when the subject is, here and now, ex- 
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And this conjecture has been re- 


periencing this particular drive. 
atory. For example, Miller 


peatedly confirmed. in the animal labor: 


(ros ib) says: 


al {chimpanzee | satiated on bananas 


Wolfe (1936) reported that an anim 
ted in them. When hungry, 


will not beg for tokens and does not seem interes 
the same animal is very much interested in getting tokens. Similarly Bruce's 
(1932) experiment suggested that an increase in the hunger drive raises the 

igl of food that rats are prevented from 
also Schlosberg & Pratt, 1956; Wike 


rewar i 
E ard value of the sight and the smell 
[3 EVA. s 
ene by the presence of a screen | cf. 
& Barrientos, 1938]. -+> 
^ In. Wolfe's experiments the 
ieee to disappear, or at least it w 
unger drive was satiated (p- 460)- | 
shich the secondary 
ock reduction could not be demon- 
that of Littman & Wade (1955). 


motivation to secure tokens 


chimpanzees" 
demonstrated, when the 


as not clearly 
Bur see also Section Vid 

lypical of those studies in W -reinforcing power 
€ R H P " 
f a stimulus associated with sh 
of shock is 


Charen E 
trated in the absence 
size as follows: 


Which these writers summi 
of reinforcement 
follows the termin l coincides with a 
1 reduction. If drive reduction 15 2 sufficient con- 
n the light should become a secondary rein- 

sociation with the drive-reduction event. If 
it should be able to reinforce an 


ction hypothesis € 


f rhe drive-redu 1 ) 
ation of shock 


A major deduction o 
Mie e A light M hich 
m ion of drive or neee 
md for reinforcement, the 
forcing agent by virtue of its 
it becomes a secondary reinforcing agents 
Instrumental act. i 

Vhis sequence of deductions W 


as rested by placing rats in a stock, giving 
them a brief shock upon the termination of which a light was flashed for one 
Second, A fter ron ER pairings. subjects were then placed ina Skinner box 
under operant veut conditions; when they pressed the bar a light flashed on. 

Animals subjected to this training program were compared for rate of bar 
pressing with animals subjected to a variety of control conditions. No sig- 
Nificant differences were found among the groups. and it was concluded that 
in so far as the deductions from drive-reduction theory were correct, the 


thieorr Á E 
Cory was not substantiated (p. 56): 


ading was obtained by Lee (1951), as re- 


Much the same sort of fit 
(1953): 


Ported by McClelland e al. 
asic aspects of motivation [ success and failure]. 

slitv of two other types W hich would result 
change. Thus the two types we have already 
Mentioned might be thought of as resulting primarily from, firs an in- 
iro pe n approach motive) and, second, frout an increase in 
Pain (an avoidance motive). Bun at least theoretically, anas anap alse: Ne 
jeans es eta in pain or ith a decrease in pleasure. One would 
Wes gine PA lead to approach behavior and the latter to avoidance 
behavior of a sort. At the present v however, there is very little evi- 


tl In addition to these tw? b 
: Vere is at least the logical pos 
rom other kinds of affecti? 


riting. 
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dence for the existence of either of these aspects of motivation, despite the 
current popularity of the notion thar stimulus reduction Is particularly. im- 
portant in motivation theory. Thus in a preliminary experiment Lee (1951) 
has shown that a cue paired with onset of shock will lead to intense avoid- 
ance behavior when presented in a new situation, whereas a cue associated 
with offset of shock will not lead to approach behavior, as it should, but to 
a somewhat less intense avoidance behavior (pp- 74-75). 


In both the Littman-W'ade and the Lec experiment the stimulus previ- 
ously associated with shock reduction was tested, for secondary-rein- 
forcement properties, in 


a “new situation," i.c, one in which shock 
had never been experienced; and the results w ere Consistently: negative. 
In Chapter 3 (p. 81), reference has already been made to an ex- 
periment in which it was shown that a stimulus just preceding or ac- 
companving shock onset acquires the capacity to act as a secondary 
punisher, whereas a stimulus just preceding or accompanying shock 
termination took on no capacity to dct as a sccondary reinforcer; and 
here the test occurred in the same situation where the training had 
taken place. However, as in the other two studies cited, the test for 
secondary reinforcement was likewise made in the absence of shock, 
Which is presumably the crucial consideration. 

Hence we may sav that the group of studies cited, of which the 
Littman-\Wade investigation is protypical, do not really test the hv- 
pothesis (here Proposed) that a stimulus associated with primary 
drive reduction will manifest secondary-reinforcing properties when 
it again occurs in the presence of tbe primary drive. Instead, they test 
—and disconfirni!—Hull’s original view that secondary: reinforcement 
is mediated bv the —$,, mechanism, independently of concurrent 
any kind (see quotation from Seeman & 
Greenburg, Chapter 4, P- 101). This type of finding is also consistent 
with the well-known finding that a 


drive or drive reduction of 


conditioned salivary response 
will occur only when the subject is hungry (Zener & McCurdy, 1939). 

The illogic of testing for the secondary reinforcing properties of a 
stimulus. which has been associated l c 
absence of shock, has been pointed. out (personal. communication) 
by Dre D T. Mason in the follo 
buzzer, which has been associated with shock termination in onc 
experimental situation will not have any very positive appeal to thc 
subject in another, neutral situation (whe 1 
experienced) for the reason that it will "remind" the subject of the 
original experimental. situation and make him less rather than. morc 
comfortable. Or, put in more precise terms, the buzzer makes the 


with shock reduction, in the 


wing way, A stimulus, for example a 


re no shock has ever been 
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second situation wore like the first one and thus promotes the ges- 
eralization of fear from the one to the other. The notion here involved 
can be further clarified by the following illustration. Suppose that 
one is sitting in a restaurant having an enjoyable, relaxed meal with 
friends and someone comes and whispers into his ear: "Don't worry! 
Now just take it casy, everything is going to be all right." Under 
other circumstances these. words might have a strongly reassuring 
effect, but under the conditions described they would almost certainly 
arouse, rather than allay, apprehension. Here they would suggest, not 
that everything is indeed all right, but that “something sust be 
wrong," because they have been heard before in situations where 
something indeed awas wrong. 
The tendency to suppose 
effectiveness of a secondary reinforcer in the absence of the primary 
drive whose prior reduction has imparted this capacity is traccable in 
large part to Hulls suggestion that secondary reinforcement is 
,—s, mechanism, which could pre- 

esence of drive and drive reduction. 


that one might reasonably test for the 


mediated by the hypothetical 
sumably operate asithout the pr 
Here, then, is at least further indirect evidence of the untenability of 
this interpretation of secondary reinforcement. 

Another reason why shock termination has commonly failed to im- 
inforcing properties to stimuli associated therewith 


part secondary re À l à 
or thirst (instcad of shock) is the 


is this. In situations where hunger t 
primary drive, the subject has to do something, after the appearance 
r before drive reduction can be experienced, 


Of the associated stimulus, à bal si 
.a stimulus may indicate that 


i.c. the subject has to cat or drink, Thus 
ailable, but the primary drive reduction does 


nod CÓ for 


food or water is now av 
not follow forthwith. The stimulus becomes, that is to s: 
the appropriate consummatory response. But w here electric shock is 
being used, the situation is different: a “safety signal” may appear and 
the shock can then go off forthwith, without the subject himself. 
g whatever about it. In this situation the signal 


having to do anvthing 

has no necessary meaning or function for the subject. It is not the 
cue for some shock-terminating response and may very well never be 
noticed or “attended to,” since the shock itself, by contrast, is so 
much more prepotent as a stimulus. Zimmerman (1957b), in discussing 
le the following comment: 


this problem, has mac 


There is a common-sense justification for the notion that a secondary 
reinforcer, in order to be effective, must be a means to an end. The color of 
r in a restaurant where we eat does not ordinarily possess much 


the w allpape : B ee 
it has little "meaning" for us. On the 


in the way of reinforcement value, i.c. 
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other hand, the taxi which we use to get to the restaurant, or the money 
c and, 2 S s : 
which we use to pay the taxi fare, does possess reinforcement value, in the 
sense that we will work to get them (p. 9). 


Therefore it would appear that better results would be obtained 
if the stimulus which is associated with shock reduction operates as a 
cuc, releasing some specific action which is instrumental in terminating 
the shock. In fact, some writers (scc Chapter 12) have generalized 
this linc of thinking and have maintained that cue function and second- 
ary-reinforcing function are inseparable, if not identical, In any event, 
it seems that, in the future, it will be desirable to require the subject 
to do something, on cuc, as a means of terminating the shock, instead of 
simply pairing some stimulus with shock termination which occurs 
without respect to the subjects behavior (scc Beck, 1957). Also it 
would seem desirable, in future experimentation. with shock-reduction 
reinforcement, to take advantage of some of the procedures which 
Zimmerman (1957b) has found so effective in maximizing secondary: 
reinforcement where hunger or thirst is involved. (sec Chapter 4). At 
the present time, a number of investigators arc working on this problem, 
with improved techniques; and there is good reason to believe that 
results will soon be forthcoming which will be less ambiguous than 
those obtained in the recent past." 


"Since the above was w ritten, Crowder (1958) has reported a series of studies 
in which the seconda -reinforcing potency of a stimulus associated with shock 
reduction was tested ij the continued presence of shock. Here, for the first time, 
unambiguously positive results were obtained, with p-values of the order of oi 
and lower, 


Also, in 1957 Carlton and Marks, using a “low ambient temperature” 
of o* Centigrade to motivate rats, found that a tone associated with a quick rise 
in temperature (provided by means of radiant heat lamps) took on unmistakable 
secondary -reinforcement properties; but here, again, the testing was done with the 
aversive drive (coldness) present, rather than absent. Hence it appears that the 


procedural difficulties previously encountered in this type of research have now 
been overcome and that we ma 


y. with considerable confidence, accept the validity 
of the foregoing discussion. This is of more than ordinary 
following reason. It has, of course, long been known that a response which is 
contiguous with shock termination is very quickly learned and occurs with in- 
creasing alacrity with recurrence of the shock. In short, shock te 
be used, with great effectiveness, to establish instrumental re 
(Chapters 2 and 3). Now as already indicated, we are here 
conception of habit in which type-2 secondary 
Chapter 7); and if it had turned out that shock termination could 
to impart S properties to a stimulus or stimulus compound 
this finding would have materially we 


importance, for the 


rmination can 
sponses or habits 
working toward a 


reinforcement. is crucial (see 


"Hot be used 
associated therewith, 
As it is, the evidence 
valid; and there is no 


akened the argument, 
is entirely as it should be, if the new conception of habit is 
cause for embarrassment or hesitation on this score. 
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V. Fear Conditioning with Hunger or 
Thirst as the UnCS 


On the basis of evidence already reviewed in this book, it is well 
established that an originally neutral stimulus, associated with the onset 
of a noxious stimulus such as electric shock, acquires the capacity to 
ssociated with 


arouse fear; and it is now equally clear that a stimulus 
the termination of a metabolic drive such as hunger or thirst will 
acquire secondary reinforcement properties. In the immediately pre- 
been explored as to whether a stimulus 


ceding section, the question ha 
associated with the termination of a noxious drive stimulus, such as 
shock, can likewise acquire secondary-reinforcement potency. The 
answer, while still somewhat indeterminate, seems to be that as soon as 
truly comparable experimental procedures are devised, it will be found 
that stimuli associated with shock termination take on this capacity, 
no less than do stimuli associated with hunger or thirst reduction (see 
footnote 8). For purposes of logical symmetry, the remaining ques- 
tion is: Can the onset of a metabolic drive be used to condition a fear 
reaction to some formerly neutral stimulus or situation, in a manner 
comparable to the fear conditioning that can be obtained, for example, 


With shock onset? : i 

Perhaps the first study that should be cited in this connection is one 
reported in 1953 by Calvin, Bicknell & Sperling. These writers pose the 
problem which they wish to investigate in the following manner. They 
say: i 
A reinforcement theory of behavior assumes that a complete behavior 
cycle may be described as instigated by a motive, or drive, and terminated by 
a reward, or reinforcement. This notion is suggested [in Fig. 5-5], which 
depicts a motive-reward cycle; this is drawn to represent a gradually in- 


creasing motive, like the hunger drive, and its rapid reduction under the 


presentation of a reward, like food. "em 
This eyele can be divided into two phases: (a) the motivating phase, cov- 
ering the period during W hich the drive is building up; and (b) the reinfore- 
ing phase covering the period of the reward, during which the drive is falling 
g phase, g 
off. 
If à previously neutr h 
of the eyele, this stimulus will 
phenoménon of secondary reinforcement (Hull, 1943), and there have been 


many experiments demonstr: ting its existence, and experimental inquiry into 

its empirical determining variables (Bersh, 1951; Hall, 1951) has begun. * 
Comparable investigations for the other phase of the cvcle—the motivating 

phase—are fewer in number. Miller (19482) has shown that a previously neu- 


al stimulus is presented during the reinforcing phase 
acquire reinforcing properties. This is the 
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tral ind presented during existence of a pain drive can acquire motivat- 
ing properties. He used the term “fear” as a name for this secondary motive 
based on pain. 

The present inv estigation was designed to determine whether a stimulus 
situation will acquire motivating properties if it is repeatedly presented dur- 
ing the motivating phase of the evele shown [in Fi ig. 5-5 |. Strength of drive 
is relatively high at point SD and it is relatively’ los. at point WD. Will a 
stimulus situation repeatedly presented at point SD acquire stronger second- 
ary motivating proper ties than the same stimulus situation repeated pr esented 
at point W D: (p. 173) 


- Motivating phase Eee 


Reinforcing i E 
SD ~ phase Fig. 5-5. Motivation-reward 


cycle, from Calvin, Bicknell, 
and Sperling (1953. ERIE 
The subjects in this experi- 
ment were tested at a point 
about midway between WD 
wo and SD, after i1 1/2 hours 


Strength of drive 


of food deprivation, 


Time in hours 


Later, the writers summarize the procedure which they employed 
thus: 


Two groups of rats were placed in striped boxes 30 min. a day for 24 
consecutive days. Group SD | strong drive| received this daily experience 
under conditions of 22 hr. food deprivation, for group WD ES eak drive |, 
the deprivation period was 1 hr. Following this procedure, the food con- 
sumption of each group was measured |in the striped box | under 11! a hr. 
food deprivation (p. 175). 


The results show that the rats which had been under strong hunger 
when placed in the distinctive box during the 24 training days ate 
reliably more when later tested, after 111, hours of food deprivation. 
than did rats which had been under weak hunger when previously 
exposed to the striped box. The authors interpret their findings thus: 


The results of this experiment indicate that a neutral stimulus | striped box | 
can acquire motivating properties if it has been associated with a primary 
drive | hunger] state (p. 174). 


The results of the Calvin-Bicknell-Sperling study thus seem to be 
clearly confirmatory of the notion that a mounting metabolic drive. 
such as hunger. can be used as the unconditioned stimulus for at least 
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a mild conditioned-fear state which, in the present instance, is indexed 
by cating. But in 1954 Siegel and MacDonnell carried out an essentially 
similar experiment, W ith completely negative results: "Our findings do 
not agree with thosc of Calvin, Bicknell, and Sperling. We can offer no 
explanation for the disagreement? (p- 251). 

In 1955, Solomon & Swanson reported a related type of investiga- 
rived of water in their home cages for 
(individually) into small “water boxes,” 
' They were then put into 


tion, in which rats were dep 
23 hours and were then put 
with free access to water for one-half hour. 
another type of compartment, quite different in shape and appearance 
from the first two. This may be termed the “neutral compartment,” 
since the animals experienced neither thirst nor thirst reduction in it. 
alf hour in such a compartment, the rats were 
there to begin another period of 
"he nature of this "circuit," which 
the rats negotiated every day for 10 days, is shown diagrammatically 
in Fig. 6. It should be added that food was always available to the 
animals in the home cage and in the water box. However, during the 
" able only in the “water box." 


After spending one-h 
put back into their living cages. 
23 hours of water deprivation. 1 


10 days of training, water was avail 


Fig. 5-6. Schematic — versi 

- . Schematic version 

of the ^e N Water Neutral 
the training procedure Home box compart- 

employed in the Solomon- cage ment 

Swanson study of thirst-fe 


conditioning. 23 hr. 1 hr. 14 hr. 


On the eleventh day of this investigation the subjects were randomly 
f and were tested as follows. The animals 
in one group, which we shall call Group E (experimental), after drink- 
ing to satiation as usual in the water box, were placed directly back 
in the home cages, where water was now, for tbe first time since the 
start of the experiment, available. The animals in the other group, 
Which we shall call Groups € (control), after drinking to satiation as 
usual in the water boxes, were put into the neutral compartment, 
where water, likewise for the first time, was available. 

If the assumptions which we have been exploring in this chapter are 


truc, how should these two groups of animals behave under the test 
As a result of repeatedly experiencing 


divided into two equal groups 


circumstances just described? 


prolonged and relatively intense t D 
should have developed thirst fear as a conditioned reaction to these 


cages. Therefore, when the animals in Group E. were put into these 
cages without thirst, thirst fear should nevertheless have been aroused 


hirst in their living cages, all animals 
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in them; and if the rats were thus afraid of being thirsty in this situa- 
tion, it would be reasonable for them to drink still more water. This 
is what they did. After having just drunk, in the preceding half hour, 
an average of 13 cc. of water, these rats, in the half hour following 
their return to the home cages, drank on the average an additional 
5.8 cc. of water. On the other hand, the animals in Group C, when 
placed in the "neutral" compartment, cach drank an average of only 
2.5 cc. of water during the same period of time. This finding is, of 
course, in conformity with the logic set forth in the Calvin-Bicknell- 
Sperling experiment; but like that of the writers just cited, this finding 
has not been confirmed by subsequent (unpublished ) repetitions 
thereof. 


In vet another, and somewhat different, attack upon the problem, 
Swanson (1955) conducted an experiment in which rats were put on 
the following 24-hour training cycle. As in the Calvin-Sperling-Bick- 
nell experiment, the rats were deprived of food for 23 hours and were 
then fed to repletion. However, they were randomly divided into two 
groups and, during the training period, were treated differently in one 
respect. Half of the rats were put into a distinctive box one hour after 
having been fed and were left there for one hour. They were then 
returned to their living cage. The other half of the animals were put 
into this box, also for an hour, but much later in the food-deprivation 
cycle (21 hours after feeding), when their hunger was relatively in- 
tense. They were then returned to their living cages. At the end of 
23 hours of food deprivation, the animals in both groups were trained 
as follows. In still another box, different from both the living cage 
and the hunger-fear conditioning box, the rats were individually 
given a brief tone, followed by a bit of food. There were 10 pairings 
of tone and food, after which the animals were returned to the living 
cage and fed as much as they could eat, thus completing the 24-hour 
deprivation-feeding cycle. This procedure was followed for 12 day's, at 
the end of which time all animals were put on full feed for two days 
and then tested in the manner indicated below. 

Having enjoyed 48 hours of unrestricted feed and having thus 
gained back any weight that they may have lost during the preceding 
12 days, the rats, now fully satiated, were put into the hunger-fear 
conditioning box with a small bar available, depression of which would 
cause the tone, previously associated with food, to sound. In keeping 
with the prediction gener: 


ted by antecedent assumptions, the animals 
that had been quite hungry when previously in the box pressed. the 
bar and produced the tone three times as much as did the animals which 
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had been satiated, or only very slightly hungry, when in the box. As 
à result of its prior association. with hunger reduction, the tone had 
presumably acquired. the capacity to reduce hunger fear. This was 
demonstrated by the fact that when hunger fear was aroused in the 
not by their being made hungry, but by being 


one group of animals, 
them of being hungry, these 


placed in a situation which “reminded” 
animals engaged in behavior that produced the tone 300*« more than 
same situation, were vot reminded by 
it of being hunerv. ( Parenthetically, Swanson had a control group of 
animals which showed that the greater bar pressing in the one group 
Was not due merely to heightened activity and exploratory tendency in 


did those animals which, in the 


that group.) 


The rats in the Swanson experiment, though satiated when tested 


in the hunger-fear conditioning box, became somewhat afraid, pre- 
sumably, that they were going to become hungry; and the tone was 
therefore a welcome sound because it said, in effect, “If you become 
hungry, vou will be fed." In short, the box reminded them of hunger 
pain and aroused fear thereof, w hile the tone “recalled” hunger reduc- 
tion and the ensuing diminution in hur 
" the procedure 
studies. Perhaps here, too, we are deal- 
sort of phenomenon. A metabolic drive in 
Comparison with a noxious stimulus, such as shock, is, after all, of very 
feeble intensity. It may therefore be that, in 
conditioning of fear with a metabolic 


drive as the UnCS is, at best, ephemeral and sometimes so slight as to 
n with more refined methods, the phenomenon 


al than the findings to date would 
gical justification; and incidental 


nger fear.” 
It is not vet known whether used by Swanson will or 


will not be substantiated by other 
ing With a tenuous, unstable 


gradual onset and rather 
light of these circumstances, the 


clude detection. Howeve 
May be found to be more substanti 
indicate, Certainly it has good lo 


a course taught by the writer, suggested 
wstructive in this connection. Primary drives may 
“death w arnings.” They say to the organism, in 
effect, Sif you expect to remain alive, vou are going 1 neen a supply of this or 
that (or, ta change this or that condition). Therefore, ioisunen Mi, Burns, 
primary repens preety serious, and it is not surprising that living organisms 
become "exercised" about them. rasa learned ur condinoned peas due 
sort of. psychological warning, not of impending death, bat pe impending primary 
drive (discomfort); and being a discomfort in its own right, Le. a secondary 
drive, the fear can stimulate. the organism to more or less appropriate behavior 

d from death dange 


While still, so to speak, one step anusa AE : 
higher-order conditioning, behavior becomes still more 


" i 
Mr. Neal M. Burns, when a student in 
pa following notion which is it 
ve thought of as physiological 


As fear, i.c, primary- 


drive danger, undergoes | 
self-preservative and “foresightful. 
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observations, to be reviewed in the next section, lend rather impressive 
empirical support, of a different kind." 

By way, then, of summarizing this section and the preceding onc, 
we may note, first of all, that experiments involving shock onset have 
been especially successful in the conditioning of fear (and the study of 
type-1 secondary reinforcement) and that in our attempts to condi- 
tion bope (and to study type-2 secondary reinforcement), experi- 
ments involving the reduction of metabolic drives have been most satis- 
factorv. Although, if our over-all scheme is sound, it is logical (indeed, 
incumbent upon us) to suppose that shock offset could also be used to 
condition hope and that fear could be conditioned by means of the 
induction of metabolic drives, these two inferences were not immedi- 
ately confirmed. However, as we now know, shock termination cai 
be used to attach hope to a stimulus or situation associated therewith 
(see especially footnote 8); and there is evidence, just reviewed, which 
at least suggests that metabolic-drive induction can likewise be used to 
condition fear. In principle, it should be possible to use the onset and 
offset of any primary drive (if sufficiently intense) to condition, re- 
spectively, fear and hope to formerly neutral stimuli. The difficulty in 
using shock offset for the conditioning of hope now seems to have 
arisen from the fact that we at first were not testing for this effect under 
appropriate conditions. And it nx y develop that some similar oversight 
is responsible for the difficulties which have been encountered in using 
metabolic-drive onset to condition fear. But here the problem may be 
more intrinsic, stemming from the simple fact that metabolic drives 
normally mount slowly, so that one cannot make any particular stimu- 
lus or situation signal drive onset in the same dramatic way that this 
can be done with shock, since the shock can be turned on quite sud- 
denly. Considering the great gradualness with which metabolic drives 
normally arise, it is perhaps a triumph for our theory that even moderate 
indications of fear conditioning have here been obtained. 

With this consolidation of theory, we are now in a position to extend 
our analysis to the phenomena discussed in the next section. 


"In this connection Miller (19593) has reported that he has “repeatedly failed 
to establish experimentally any appreciable learned elicitation of drive on the basis 
of primary drives such as hunger or thirst" and cites a study by Myers and Miller 
(1954) as an example. “Although this failure has not caused me to 
pletely the attempt to apply the fear-clicitation paradigm to 
has caused me to shift the emphasis in a new direction." 
of the possibility that “the primary drives themselves m 


to abandon com- 
appetitive drives, it 
namely, in the direction 
ay be profoundly modified 
by learning" (p. 263). It remains to be seen what fruit this new hypothesis will bear. 
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VI. Hoarding, Externalization of Drive, 
and Related Phenomena 


The experiments cited in the two preceding sections, prompted as 
they were by contemporary theoretical formulations, have all been 
carried out and reported quite recently. There are, however, a 
number of older observations and laboratory researches recorded in 


the literature which can now, in light of these formulations, be more 


factorily explained than was formerly possible. 

first of all, the phenomenon of hoarding. Here rats, 
will nevertheless continue, sometimes for a 
to go for pieces of food at some distance and 
cage." This phenomenon has been 
various instinctual and 


Let us consider, 
though food-satiated, 
surprisingly long time, 
carry them back to the home 
extensively studied in the laboratory, and 
s of explanation have been put forward to account 
for such behavior (Morgan, 1947). all with some measure of empirical 
support. However, the interpretation which seems most natural in light 
of the preceding discussion goes as follows. Even though well filled 
With food and thus free from hunger, a rat may nevertheless be in 
the presence of certain stimuli which cue off hunger fear (or some 
other fear; sce Section VID); and the latter, rather than hunger proper, 
may motivate the characteristic hoarding pattern, because the stimuli 
involved in such behavior have hunger-fear reducing (secondary 
This point of view has been most fully and 


physiological type 


reinforcement) properties. 
Ingeniously developed ina p 

Or, in order further to suggest t 
ing and the type of thinking developed it c r 
let us revert to the Solomon-Swanson experiment. Here rats, after 
being repeatedly deprived of water in a particular cage, were satiated 
and, upon being returned to this cage, drank still more water. In a 
sense, it would be fair to say that they were boarding water. They 
were, we infer, afraid that they were going to become thirsty; and 
ave seen, would have a tendency to reduce such 


aper by Marx (1950). : 
he relationship between food hoard- 
1 other parts of this chapter, 


water drinking, as we h 
f shenomenon has been cited by Miller (1951b). as fol- 
d hungry dogs to lift their paws to get food When 


d to respond longer if he kept 


An apparently simil 


lows: “Brogden (1942) traine 4 
the dogs were satiated he found that they continued t , 
lid not eat) than if he disconnected the food- 


giving them food (which they d : 
some learned drive must have been present to 


delivery mechanism. Presumably » à 
make the sight of food effective as a learned reward in the absence of hunger" 


tP- 461; cf. also Myers, 1949)» 
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fear. Rats with hunger fear (but no real hunger) can cither eat or store 
food in a corner of their cage or nest, whereas the only place the rats 
with thirst fear can "store" water is—in the manner of the desert 
camel—in their stomachs.'? However, the underlying mechanisms in 
the two situations seem to be basically similar. 

Thus we see how the motivation for hoarding behavior presumably 
arises; but are we equally clear as to how such behavior gets reinforced? 
For this form of behavior to persist, it must be in some w ay uniquely 
satisfying; i.e., it must make the animal feel better than anything else 
it can do. How is such an effect achieved? Following a line of reason- 


ing dev eloped earlier (see especially Section II), we may assume that 
rats are fami 


ar with zwo "situations" or “conditions” which may 
prevail in their home cage (or, in a state of naturc, in their den or nest): 
cage-with-food-present and cage-without-food. Strong hunger is never 
experienced in the first situation; whereas it may well be experienced 
in the second one. Hence we may assume that hunger fear tends to get 
conditioned to empty cage, whercas no such emotion is aroused by 
cage-with-food. Now let us suppose that a rat, though not immediately 
hungry „isinan empty cage. Qur assumption is that here he may well ex- 
perience hunger fear. Suppose, also, that a supply of food is available 
at a little distance from the home cage through a runway connected 
to the cage. By going and fetching food from this remote supply and 
putting it in the home cage, the animal manifestly changes empty-cage 
into cage-with-food and, in the bargain, reduces his conditioned hunger 
fear. The resulting diminution of fear presumably reinforces (rew ards) 
the food-fetching behavior which we call “hoarding.” 

This interpretation g gains support from the following statement by 
Stellar (1951): 


(1) Rats do not carry any appreciable amount of food into their cages 
until after they have been kept on a food deprivation schedule for a number 
of days, and (2) once started, hoarding does not stop until after rats have 
been allowed continuous supplies of food for several davs. Both of these 
findings have been amply substantiated (Morgan, 1947). 1 wey make it clear 
that hoarding does not depend upon simple hunger or on any other acute 
effects of food deprivation. Rather, hoarding seems to be the result of some 
effect produced by repeated deprivation and rectified only by 
periods of satiation (p. 290). 


fairly long 


© Bindra (1947), interestingly enough, has shown that, under certain conditions, 


rats will hoard cotton balls soaked in water, though not at the moment thirsty. 


His conclusion is simply that rats will hoard anything which they are likely to 
need, 
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Although some more basic physiological mechanism may be in- 
volved here, certainly the facts cited by Stellar are congruent with the 
notion that hoarding is based upon hunger fear and the reinforcement 
mere presence of food (cf. Lowney, 1958). 

i ave reported a study the purpose of which 
was “to determine whether or not a secondary reinforcing agent 
of food] will facilitate runway performance 


s of hunger and thirst are satiated” (p. 441). 


provided by the 
Wike & Casey (1954) h 


[sight, smell, and touch 
when the primary driv 
And in conclusion these investigators say: 


Since the experimental Ss, traversing a runway to an end box containing 
food, ran more rapidly ad consistently than the control Ss running to an 
empty goal box, it wi concluded that the sight of food has reward value 


when the primary drives of hunger and thirst are satiated. y 
The learned reward value and its observed persistence were attributable to 


the historical and contemporary association of the learned reward with drive 


reduction (p. 443)- 


a kind of “hoarding. But instead of the subjects 
and returning it to à nest box, they took them- 
as an abundance of food, rather than 


Here, too, was 
going and getting food 
Selves to a place where there W 
as none. 
tendencies? Rather obviously they 
having “food stores” as well 


remain in a place where there w 
how similar 


Do human beings $ 
at all possible, upon 


do. They insist, if it is 
as various other kinds of 
“hoards,” are usually conim stores, 


use them. But we want such a store 
in the sense of a building. but as a store or reserve supply of the 
When. during a war, there is a shortage of 
certain “goods” (drive reducers, reinforcers) in common, public 
a for families to try to establish their own 
is also called “hoarding” and is 


or supplies, or 


"stores." True, these stores 
i.c., several different families will 
to be regularly available, not just 


Commodity in question. 


stores, there is a tendency 
private “stores.” This, of course. d es 
officially discouraged, since more of the commodity is now needed to 


tc stores than one common store; but “hoarding” 
rmal situation, the main difference being that in 
reserve is private, in the other shared, 
Common, As long as an abundance of food is in sight, cither in a pub- 
& ` there is no hunger fear; but if there is food in 
al, though not immediately hungry, may 


stock the many priva 
Is of a piece with the no 
the one case the reassuring 


lic ora private “store, 


neither place, the individu à A 5 E 
about where his "next meal is coming from. 
re has been a good deal of interest in, though 


ng of, a phenomenon known as the 


still be worried 
Over the years the i 
apparently some misunderstandit 
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“externalization of drive.” A classical example of this sort of behavior 
was reported by E. E. Anderson ( 1941€). First of all, this investigator 
gave 20 hungry rats onc trial per day for 73 days in a 14-unit 'T-mazc. 
which he called Maze-A. A "rest period" of two wecks, on full feed, 
was then provided, after which the subjects were again put "on drive” 
and divided into four different groups, of five animals cach. Group 


1-HR (hungry, rewarded) was given onc trial per day for 37 days in a 


-unit T-maze (with a pattern different from that found in Maze-A). 
The results are indicated in Fig. 5-7 by the solid curve. Since the maze 
was so short and simple, these subjects, “normally” motivated (by 
hunger) and rewarded (by food), learned Mazc-B quickly and well. 


Errors 


1 2- 5- 8- 1l- 14- 17- 20- 23- 26- 29- 32- 35- 
4 1 JV 3$ 3& 39 22 qs 28. 3|. 34 37 
Trials 
Fig. 5-7. Learning curves for four groups of rats, run through à maze under four 
conditions of motivation and reward described in text (Anderson, 1941€, p. 402). 


The animals in Group 2-HNR (hungry but nonrewarded) were 
treated just like the animals in Group 1-HR, except that they received 
no food in the goal box, at the end of the maze. The Group 3-SR 
animals (satiated but rewarded) and the Group 4-SNR animals 
(satiated and nonrewarded) were treated just like those in Group 
1-HR and Group 2-HNR, respectively, except that they were not 
hungry (having been fed shortly before the daily maze run). The 
performance of the animals in Groups 2-HNR, 3-SR, and 4-SNR 
are also shown in Fig. s—7. 

These findings make sense. The animals G-HR) which were 
rewarded, on completing the maze, by reduction in both hunger and 
hunger fear. learned best. The next-best learners were the animals 
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(2-HINR) which were motivated both by hunger and by hunger 
fear but which were rewarded only secondarily, by hunger fear 


reduction. And, as the secondary reinforcing potency of the goal- end 


of the maze extinguished, their performance tended to deteriorate, as 


one would expect. 
Elimination of hunger as a primary drive caused the performances 


of the animals in Groups 3-SR and 4-SNR to be decidedly inferior 
to those of the first two groups. The satiated animals were, presumably, 
motivated largely o r perhaps exclusively only by such hunger fear as 
had become mad to the maze (see Section Il); and, quite 
understandably, it did not matter much under these circumstances 
Whether the subjects did or did not get a bit of food (which they did 
Not especially need) at the end of the maze. These two groups, in other 
words, ran and learned Maze-B, to the extent that they did run and 
learn it, on the basis of secondary motivation and secondary reinforce- 


ment alone (see also Fig. 5-8). 


rI 
4 
3 
" 
iz 
a? 
1 
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47- 53- 59- 65- 


33- 29- 35- 4l- 
ious CU We H- D- $ oa s * 8 GU 


Trials 
Fig. 5-8. Learning curves for four groups of rats intended as "controls" for the 
= (Anderson, 19416. 


fc 2 ». 408). 
"Ur curves shown in Fig. 577 pt 


to the. Anderson study than here re- 


ported, and it should be examined in the context of two carlier papers 


by the same author (Anderson, 19412. 1941b) for a full appreciation of 
and logical wav in which it was carried out. How- 
of the Anderson study need not be further 
ally confirmed, in a different 


There is a good deal more 


the very methodical 
ever, the empirical aspects O 
reviewed here, since it has been dramatic 
sort of study reported, by Danziger (1951)- 
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The only question, it seems, is one of interpretation. Both Anderson 
and Danziger spoke in terms of the “externalization of drive"; and, 
by implication, they were thinking of the hunger drive. The alternative 
assumption here favored is that hunger, like strong primary drives in 
general, arouses fear and that this reaction can become conditioned to 
stimuli or situations which have been associated with the onset (or 
augmentation) thereof. Thus, instead of speaking of the "externaliza- 
tion” of the hunger drive, one may refer more appropriately to the 
subsequent arousal of hunger fear (which, of course, is an internal 
state) by an external stimulus or situation, And, of course, the elimina- 
tion of such stimulation would be expected, in keeping with principles 
already elaborated, to have a (secondary) reinforcing effect (cf. 
Denny & Bchan, 1956) i ) 

Earlier in this chapter (Section IV) we arrived at the conclusion 
that secondary reinforcement type-2 can be demonstrated only if the 
relevant primary drive is currently present. Now it seems that, in the 
present and immediately preceding sections, we have a good deal of 
evidence to the effect that this phenomenon can be found in situations 
where there is complete satiation as far as primary drive is concerned. 
Fortunately, this paradox is easily resolved by our theory. This theory 
holds that secondary reinforcement can be demonstrated cither if the 
primary drive is present or if there is fear of the recurrence thereof. 
If, for example, hunger fear is being stimulated by hunger itself, then 
a stimulus which promises hunger reduction will reduce the hunger 
fear; but, similarly, if hunger fear is being aroused on a conditioned 
basis, in the absence of hunger, the elimination of the conditioned 
stimulus or situation will likewise act as a secondary reinforcer. ‘The 
first is clearly an instance of ty pe-2 secondary reinforcement, whereas 
the latter qualifies as secondary reinforcement ty pe-1. Our. assump- 
tion is that type-1 secondary ‘reinforcement occurs when a fear of 
drive Increase (of any kind) is disconfirmed and that type-2 secondary 
reinforcement occurs when there is a prospect of drive decrease (see 
also Sections I and IES. 


VII. Drive Fear and Drive Equivalence 


The fact that a response learned under the motivation and reward 
provided by one source of drive may later “crop up" when the organ- 
ism is ostensibly motivated by quite a different drive has occasioned 
a good deal of interest and at least mild perplexity. And Estes (1949) 
has reported an experiment in which it was shown that a stimulus which 
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had been associated, in thirsty rats, with thirst reduction could be 
used to reinforce a Skinner-type b: r-pressing habit when the rats were 
not thirsty but bungry. Estes concludes from this finding that "since 
the effectiveness of a secondary reinforcer clearly is not specific to the 
original drive, it will not be profitable to define the concept of re- 
inforcement in terms of drive reduction" (p. 294). The present writer 
ggested the following alternative. interpretation: 


has previously su 

It will be remembered that the subjects [in the Estes experiment] which 
were tested when satiated with water were strongly motivated by hunger. 
Thus, although there was no fear of thirst in them, there was, presumably, 
now fear of unger instead. It is by no means far-fetched to ume that a 
stimulus which has become capable of reducing thirst fear will also be capable, 
through generalization, of likewise reducing hunger fear. Granted these in- 
ferences, there is nothing inexplicable about the fact that a stivulos which 
had previously been a sociated with thirst reduction was later capable of 
providing secondary. reinforcement, even though the subjects were, at the 


time, warer-satiated (Mowrer, 1959. P- 149)- 


re is that the “equivalence” sometimes observed 
ry drives stems from the element or component 
to these drives. This thought is obviously re- 
‘a generalized drive state” or drive generality. 
perimental literature, Grice & Davis 


The assumption he 
between different prima 
of fear that is common 
lated to Hull's notion of * 
Summarizing the relevant. CX 
(1957) have recently W ritten as follows: 
activating character of drive (D) which has 
) and Spence (1954) has received a fairly 
substantial amount of empirical support. Amsel (1950) found that rats will 
perform a running habit mouy ated by fear more vigorously when they are 
also hungry. Amsel and Maltziman (1950) found that the presence of emo- 
tionality will, under certain circumstances, increase the amount of drinking. 
The lowa studies of anxiety in human Ss have been based on this imption 
and, in general, have y ielded results in accord with prediction (Farber and 
Spence, 1953; Montague, 1953: Taylor, 1951) [Sec also Spence, 1958]. In 
the case of purely appetitional drives, Webb (1949) trained rats on a panel- 
: food deprivation for food reward and conducted 
arious degrees of water deprivation with the hunger 
drive satiated. He found that the number of extinction responses was an 
iner number of hours of water deprivation. This result 


sing function of the A "milarl s ^ 
was confirmed by Brandauer (1953) in a similarly designed experiment em- 


ploying a bar-pressing response (p. 347)- 


The generalized response- 
been hypothesized by Hull (1943 


pushing response under 
extinction trials under V 


Elsssvercone complicating consideration to which Grice & Davis 
point is that there is some evidence that in a number of laboratory 
animals, and notably rats. thirst and hunger appear to be somewhat 


interdependent: i.e. a hungry animal will not drink as much as a non- 
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hungry one and a thirsty animal will not eat as much as a nonthirsty 
onc. The possibility therefore exists that the satiation of cither of these 
drives while the other one is strong is questionable. When special 


provisions were taken to control for this sort of interdependence, 
Grice & Davis (sce also Ellis, 1957) failed to confirm the Webb find- 
ings, cited above. 

However, there are other studies of drive interaction or summation 
where the same mechanism can hardly be suspected of operating. For 
example, Braun, Wedekind, & Smudski (1957) have studied the sum- 
mational tendencies of hunger and discomfort produced by immersion 
in water (rather than thirst). They preface their study with this the- 
oretical statement: 


Hull (1943) has made the assumption that response strength is a ier oe 
cative function of learning or cognitive factors and motivational or drive 
factors. Drive factors are “design: ated by the hypothetical construct, drive 
strength (D), which comprises the total effective drive strength operating 
in S at any giv en moment. In any experimental situation, both relevant and 
irrelevant needs or motivations arc assumed to sunmmiate to contribute to 
the value of D. A relevant need is one which is reduced by the response 
under consideration; irrelevant needs encompass all other primary | and sec- 


ondary needs in S when the response occurs but which are not reduced by 
the response (p. 148). 


And these inv estigators summarize their empirical findings thus: 


This study inv estigated the learning by the rat of a modified Lashley Maze 
IH adapted for sw imming asd function of the combination of a relevant drive 
—escape from water— —and an irrelevant driv e—hunger resulting from food 
deprivation. Two levels of relevant drive were produced by using water 
temperatures of 35°C. (high) and 15°C. (low). The irrelevant. drive was 
induced by o-hr. EU 22-hr. deprivation, The addition of a high irrelevant 
drive to both high and low levels of relevant drive results in an increased 


total drive strength as measured by swimming time and error elimination 
(p. Ys). x 


Even more striking is an experimental finding reported by Ullman 
(1951). In this studv, 


Fight female rats were taught to eat small pellets of food in an apparatus 
whose floor was capable of being electrified. After four davs of 20-minute 
experimental sessions during w hich the time when cach pellet of food was 
eaten was recorded, electric shock of moderate intensity was introduced for 
five seconds of cach minute. This led to a reduction in cating for the first 


two davs, followed by an increase in eating especially during the five seconds 
when shock was on. Then the intensity 


of shock was increased for four more 
days. During this period, eating activity increased markedly, with a vers 


large increase during shock. For the last four davs of the experiment, the 
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animals were satiated prior to being placed in the apparatus, and the pro- 
cedure was carried out as before. During this time, total cating activity de- 
creased, but most of the eating was done during the time when electric shock 
was on (p. 581). 

Before experimentation proper was begun (as a sort of control) 
all ei 
cach dav for a period of four davs. Hunger was standardized, with 


rhe subjects were allowed a period of 20 minutes of “free eating” 


animals being held at 80'£ of normal body weight. Figure 5-9 shows a 
composite bar diagram for the total number of pellets eaten, by fivc- 
second intervals, during the free-cating periods on these four days. 
As was to be expected, about the same number of pellets were con- 
sumed during cach of the 12 five-second periods. In all. the cighr rats ate 
; g the four 20-minutc periods. 

5 g pattern for the cight rats used in this 
study when a mild electric shock was put on the floor grill during 
the first five-second interval in each minute. (When the rats were first 
put into the apparatus, instead of being shocked immediately, they 
were allowed cach day a one-minute orientation period without 
shock.) Hunger was held at the same level as before, cach subject had 
20 minutes cach day in the apparatus, and this part of the investigation 
again lasted for four davs. As will be noted in the bar diagram, the 


number of pellets caten durir 
slightly, but not markedly, gre: 
period. It should also be said that the tot 
so-minute periods was 
four periods of free 


2,305. pellets duri 
Figure 5-10 shows the catin 


ng the five-second shock interval was 
iter than during any other five-second 
al number of pellets consumed 

3.049. 25 compared to 2.305 
for the preceding opposed to "forced") 
cating. This increase of 744 pellets may have been duc to unidentified 
causes (e.g, greater familiarity with the apparatus, or increased 
Sire of body weight). But it could also be in- 
act that the introduction of the shock 


ation by making them somewhat 


during the four 


hunger despite constancy ) 
terpreted as arising from the f 
increased the subjects total motiv 
further credence by the next phase 


fearful. This suggestion is given 


of the study. 


Because the increase 1n € ! c | 
(in. comparison with results obtained in a pilot study), shock in- 


tensity was somewhat increased during another four-day experimental 
are shown in Fig. 5-11- Here we see that the 


ating produced by the shock was so slight 


period. "The results 
number of pellets caten during the 

Sane hok s T ROR BATADIC seval. 
as great as the number eaten during any other comparable interva 


Moreover, the total consumption of pellets rose to à new high of 3.630. 
‘The next and final phase of the experiment consisted of four daily 


shock interval was more than twice 
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20-minute sessions in the apparatus after "complete" satiation of 
the subjects on wet mash (a preferred food) in their living cages, 
just prior to being put into the apparatus. The very interesting results 
thus obtained are shown in Fig. 5-12. As was to be expected, there was 
a marked over-all drop in the number of pellets (530) consumed; but 
still caused a great augmentation in eating 
and that, in the eleven five-second intervals between shocks, there 
was still some cating, despite the immediately prior “satiation” of the 
animals. The presumption is that this cating was motivated, purely and 
simply, by fear? 

Ullman gives an astute anal 


it is noteworthy that shock 


ysis of his findings in these words: 


an immediately be seen from the 
an extrancous stimulus, electric 
ag behavior leads to an incre: 
" cc. inter- 


The salient feature of the experiment ¢ 
results described above. The introduction of 
shock, after a brief dep ession in pellet-takir [ 
in this activity, an in se which is particularly marked in that 5- 
val during which the electric shock is on. . 

In analyzing what occurred in the experiment, we see that along with the 
two sources of primary motivation. hunger and pain, from clectric shock, 
two derived drives can be posited. These may be called “hunger fear and 
"shock-fear." The assumption is here made that any primary drive has a 
coexistent emotio component. Perhaps this emotional drive takes the 
form of fear of more of the primary drive; thus, not only are the rats hungry, 
but they have learned that they may go on being hungry. It is as if hunger 
were a conditioned stimulus for more hunger. In the same manner, a fear of 


reresting if Ullman had preceded the shock with a five- 
luring this period would presumably have been 
“of the inter-shock interval, though not so 


"Te would have been in 
second warning signal. Fating c 
considerably higher than in the rest o 
high as when the shock itself was on. 


LEGEND for Fig. 5-9. 
Fig. 5-9. Bar diagram sh 
during each s-second interv 
1951, p. 578). were obtained. during 


owing uniform eating, on the part of eight hungry rats, 
al in one-minute cvcles. The data (taken from Ullman, 


four 20-minute test periods on successive days. 


LEGEND for Fig. 5-10. 1 1 
Fig. 5-10. Bar diagram of cating during 


mild shock during the first five-second 


four 20-minute daily test periods involving 
interval in each one-minute cycle. 


LEGEND for Fig. 5-11. 
Fig. 5-11. Dar diagram of catir 
a somewhat stronger shock 


ag during four 20-minute daily test periods involving 
during the first five-second interval in each onc- 


minute cycle. 
LEGEND for Fig. 5-12- 


Fig. 5-12. Bar diagram of € d 
escribed for Fig. 5-11, 


ating during four 20-minute daily test periods under 


3 TA CC hat the subjects were E 
sanie-conditions as: di except that the subjects were not hun 


gry. 
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shock may be developed. The entire stimulus situation, the apparatus, the 


experimenter, etc., may be considered a conditioned stimulus which becomes 
paired with electric shock (p. 580). 


Ullman then considers various wavs in which this complex motiva- 
tional structure may have operated in his experiment but concludes 
in favor of the following interpretation: 


What probably happened was that the animals had learned to reduce 
hunger-fear by eating and that this behavior, through generalization, was 
elicited when pain-fear was created. The only adjustive act available to the 
animal was eating, and this act served to reduce the entire burden of tension 
or drive from all sources except pain. Thus, three of the four ongoing drives 
were being reduced by eating: hunger, hunger-fear, and pain-fear. This 
point of view is consistent with the results obtained in the satiation procedure 
during which one of the two primary drives, hunger, is absent, and it is pre- 
sumed that its emotional component, hunger-fear, is also eliminated. [cf. 
next experiment to be described]. Then, when the animal eats to shock, it 
is reducing only pain-fear, something it learned to do in the earlier pro- 
cedures of the experiment (pp. 580-581). 


The Ullman experiment is reminiscent of a clinical phenomenon 
sometimes referred to as “ice-box neurosis.” Here individuals, when 
they become anxious or tense, use eating as a way of dealing with this 
emotional state (see, for example, Bruch, 1943; Greene, 1939; and 
Science Illustrated, 1946). Perhaps, for reasons which will be considered 
in Chapter 6, the stimulation of the taste. (and olfactory’) receptors 
acts in such a wav in these individuals as to directly assuage fear; but 
there is also the possibility that food, which by long experience has 
been found to reduce hunger and hunger fear, now also, by generaliza- 
tion, is used to reduce other fears." 

It is, of course, only to be expected that when an "irrelevant" drive 
exceeds a certain intensity, it will have a disruptive rather than fa- 
cilitative effect upon behavior appropriate to the so-called relevant 
drive. And there is now considerable laboratory. evidence for this sup- 
position. Recently it has been shown ( Chapter VI) that anxiety in 
human subjects, if too intense or if the task at hand is delicate and 
complex, may impede rather than expedite performance. A similar 
relationship is also known to exist with respect to what was formerly 
termed "dynomogenesis" (Morey, 1933; Brown, 1955). 

And Froféeva, working in Pavlov's laboratory, reported that. an 
electric shock of gradually mounting intensity could be successfully 


" An overweight character on a recent TV program put the theory well when 
she said: “Whenever I get upset, I always have to nibble.” (Cf. Miles, 1958. 
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used as a stimulus for conditioned salivation, wtil a certain point was 
reached, at which the dog's behavior became completely disorgan- 
ized—or, more exactly, re-organized and directed toward the shock, 
rather than toward food (Pavlov, 1927. PP- 289-290). 

is to be inferred, systematically, from this large and 
heterogeneous. mass of research. findings and commonplace observa- 
tions? The fact of drive interaction, generalization, equivalence, or 
and it is consistent with the facts to 
suppose that this phenomenon is achieved by virtue of the element of 
fear which seems to be common to all primary drives, when they 
reach moderate to great intensity. A special application of this sup- 
position is made elsewhere ( Mowrer, 1960, Ch. 10); its more general 
implications will be developed in Chapter 7. Alternative interpreta- 
tions of drive equivalence and summation are, of course, possible and 
even plausible; but thc interpretation of greatest generality and syste- 


Matic power seems to be the one her 


So what, then, 


summation seems unmistakable; 


e outlined.” 


VIII. Hope and Disappointment: A New Kind 


of Secondary Punishment 
scussion is an implication of no little 


Inherent in the foregoing di: i 
theoretical, and even some practical, interest. In this and the preceding 


chapters we have seen how, step by step, it has been necessary to modify 
a strictly behavioristic approach to the psychology of learning by the 
introduction of intervening The first of these was the con- 
cept of fear and the attendant. concept of fear reduction, relief, or 


variables. 


*A particularly significant line of developments has just been reviewed by 
Malmo (1958). Noting the un 
operational definitions of drive, 


precise and generally unsatisfactory nature of most 
Malmo (on the basis of a now growing literature) 
suggests that degree of emotional arousal (as indexed by the PGR or other 
physiological nicisürés) may yet turn out to be the best measure of the intensity 
of any given primary drive. He says: "Drive s.s has traditionally been inferred 
0 h as hours of deprivation, strength of shock, and other such 
2. It appears that what we have been calling ‘arousal’ 
tical with the concept of general drive, or D." 


from conditions such 


antecedent. conditions. - 
ing iden 
particularly. relevant study by Freeman (1940), Malmo 
ance appeared to serve in place of con- 


comes very close to be 
And in his reference to 3 p? 
"In. other. words, palmar conduct à to 
Ventional conditions (egs time since last eating, or drinking) as the independent 
variable in this experiment? (p. 245): In short, Malmo is proposing that hat 
we have in this book described as drive fear (or "emotional arousal") is not 

in fact, be our most direct and reliable index 


only a real phenomenon but may, i 
of drive itself. If this way of thinking is substantiated, it will provide powerful 


additional support for our 


says: 


general position. 
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what has here been termed secondary reinforcement tvpe-i. In. the 
preceding and present chapters we have also seen how essential it has 
become to acknowledge the reality of ty pe-2 secondary. reinforcement, 
which is clearly equivalent to what is ordinarily called bope. ‘The 
latter step, as we have seen and shall further sce, resolves a great many 
otherwise vexing problems and lavs the basis for a new conception of 
habit which has great generality. But, in passing, attention should like- 
wise be called to the fact that just as fear implies relicf, so also docs 
hope imply disappointment, a concept familiar enough in common 
speech and experience but only just beginning to be recognized scien- 
tifically (scc Chapter 11). 

Clearly enough, if ¢ signal has acquired the capacity to arouse hope, 
i.c., promises relief from some primary discomfort, and if such a 
signal is presented and then withdrawn before it is confirmed, there 
will be a recrudescence of fear, dismay, hoplessness. ‘The individual, 
as one may sav, "loses hope," fecls disappointed. And this, it would 


seem, is precisely what happens when a type-2 secondary reinforcer 
is presented and then withdrawn, without fulfillment. And from this 
reasoning yet another inference follows. If fear induction, produced 
by appearance of a danger signal, can act as a secondary punishment, 
then the fear induction mediated by disappearance of a so-called 
safety signal should likewise have punishing properties. Said otherwise, 
disappointment is punisbing, as common experience amply confirms. 
But the only attempt which has apparently been made to date to bring 
this inference to experimental demonstration is a study briefly re- 
ported by Morse & Herrnstein (1956) under the title: “The Maintec- 
nance of Avoidance Behavior Using the Removal of a Conditioned 
Positive Reinforcer as the Aversive Stimulus.” Unfortunately, the pro- 
cedure employed in this investigation is intricate and its full under- 
standing would presuppose knowledge of several matters which will 
not be considered until later, How ever, Professor William E. Kappauf 
(personal communication) has suggested an experiment which is both 
simple and clear. For a thirsty rat the appearance of a small tube from 
which water can be slowly drunk acts, in effect, as a promise (of thirst 
reduction) and its withdrawal (before thirst is eliminated) presumably: 
produces disappointment. Now suppose that for a second or two 
before the tube is withdrawn a tone or some other distinctive stimulus 
is presented. Certainly the still thirsty rat will not “like” to hear this 
tone, signifying as it does the imminent disappearance of the water 
tube; and it seems likely that if, during the period while the rat is 
Ww aiting for the original appearance of the tube, it could produce the 
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tone by pressing a little bar, it would make a point of not repeating 
such an action, once it had occurred. In other words, the tone would 
very likely punish any action which preceded it, just as a hope-arousing 
signal has the reverse effect (cf. ZbroZyna, 1957, 1958; Ferster, 1957, 
1958). 

A new systematic 
are two forms of secondary 
‘there, apparently, two forms of secondary 
rit has already been pointed out that 


relationship thus comes into view: just as there 
reinforcement, which we have designated 


types i and 2, so also are 
punishment. In an earlier chapter 
an action which is followed, not by p 
by a “warning,” produces what may 
punishment; and now we may appropriately designate so-called disap- 
pointment as secondary punishment type- Thus, type-i secondary 
punishment is the antithesis of secondary reinforcement. type-1; and 
type-2 secondary punishment is the antithesis of type-2 secondary 

: are two types of secondary 


reinforcement. Or, one may say, there 
1 two types of secondary incremental 


rimary punishment, but merely 
be termed type-1 secondary 


decremental reinforcement anc 
reinforcement. 


ior, Skinner (19533) considers the effects on 
removing positiv » and negative reinforcers—a 
The effects of some of these operations 
iallv those which serve to define 
and Herrnstein, personal com- 


In Science and buman be 
behavior of presenting and 
sort of four-fold contingency boi 
are well-substantiated experimentally, espe 
positive and negative reinforcers (Nlorse 
munication). — — 

The suggestion is strong that 


positive secondary reinforcement figures 
in the explanation of "joys"—like the joy of revisiing the scenes of our 
childhood.” Apparently the source lies in the reinforcements connected 
With familiar. places people, things, and actions. The loss or absence of 


old secondary reinforcers, on the other hand, seems often to be at the 
heart of our "sor " |. Indeed, the tie-up of secondary reinforce- 


Ment and “feelings whole lives as social organisms. . . . 
"The changes in rat behavior resulting from stimuli that have been regu- 
larly Soeiired with electric shock often lead observers to attribute fear 
to these animals. The sudden burst of bar-pressing and biting responses 
an extinction S sion is frequently said to show anger. 

Ve now see that the use of these two terms 1$ justified in some degree. 
Sorrow and joy, on the other hand, are seldom imputed to the rat in any 
situation, even by persons W ho regularly employ such terms in connection 
With the behavior of dogs, chimpanzees, and human beings. Are there 
Operations, different from those we use to define anger and fear, which 
are related to special changes in reflex strength? (Keller & Schoenfeld, 1950, 


P- 256 and p. 344.) 


l'OWS. 
seems to pervade our 


at the beginning of 


You will recolleet that Shand (1914) named an important group of human 
“prospective emotions of desire.” 


emotions, e.g., hope and disappointment, 
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and that to these McDougall ( 1919) added a group of retrospective emotions, 
e.g., regret and remorse. Few psychologists except Stern, who included a 
chapter on “The Temporal Reference of Feeling” in his General Psychology 
(1938), have paid much attention to the Shand-MeDougall Scheme. Yet e 
planations in terms of physiological concomitants and expressive movements 
have not taken us far in the understanding of the more complex human emo- 
tions. Temporal reference may, after all, hold the key to a more fruitful 
interpretation (Hearnshaw, 1956, p. 18). 


I say there be four perturbations of the mind, desire, joy, fear, sorrow; 
and whatsoever I can dispute thereon, by dividing cach into its subordinate 
species, and by defining it, in my memory find I what to say, and thence do 
I bring it (St. Augustine, 399, peer" 


Our “rediscovery” of phenomena which, it seems, are as old as 
mammalian life itsclf may not seem like much of an accomplishment; 
and certainly there is every reason to be appropriately modest about 
it. But in one respect, there is real progress here: these phenomena 
are now being identified in a systematic conceptual framework and 
their definition is in terms of clear-cut, empirical operations. In this 
small but not insignificant way only does the present type of analysis 
represent any very substantial advance over common sense. 


IX. The Situational Definition of Emotions 


Since the preceding sections of this chapter were written, a par- 
ticularly relevant paper by Hunt, Cole, & Reis (1958) has appeared 
which calls for special comment. ‘These writers begin by listing several 
criteria which have been used in the past for identifying the emotions: 
"(1) the overt responses or expressions, (2) the inner organic, neural, 
and visceral changes or indicators, (3) the arousing situations, (4) the 
emotional experiences, and (5) the motivational cffects” (p. 136), of 
which the third is singled out by these authors as the most valid and 
useful. They say: i 

Arguing in this same direction, Sherman (1927) has found that judges 
attempting to name the emotions expressed by young infants show very poor 
agreement when they see only the overt behavior of the infant, but the 
percentage of agreements jumps sharply when they are shown moving pic- 
tures which include the situation. Moreover, when the judges base their nam- 
ing upon movies in which stimuli and the expressions have been transposed, 
it is the names based upon the situations which prevail (p. 137). 

W. A. Hunt (1941) has advocated standardizing the stimulus-situations 


for all emotions investigated in a manner similar to that done for the startle- 
response. Before this excellent suggestion can be implemented, however, 


" Cf. also passage quoted from McClelland er al, pp. 141-142. 
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more Ay Neer or : ¢ ea ; 
re must be known about the general charactristics of the situations which 


serve as cues for A i in i 1 
‘ ve as cues for the various complex emotions. The burden of this paper is 
G Den > a: : ; 

present two hypotheses concerning the characteristics of the relationship 


be w "i 1 1 ivi 1 i i i i 
tween the reacting indiv idual and his environment which differentiate the 
er, and sorrow, and to 


three x 4 
eR unpleasant emotions commonly termed fear, ang 
escribe several tests of these hypotheses (p. 138). 


employed by Hunt, Cole, & Reis 
s arily concerned with the 
fact that these investigators point the way so clearly to the importance 
l. val, definition of emotion, which is 
preceding discussion, in this and 
relief, hope. and disappointment 


Fhe details of the procedures 
cannot be reviewed here; instead, we are prim 


of using a situational, or operation 
What has been tacitly done in the 
earlier chapters. All emotions—fe: 
—have been identified in terms of the situations that presumably 
evoke them, and not in terms of subjective descriptions, physiological 
Hunt, Cole, & Reis, on quite inde- 
roach and give us added 


concomitants, or the like. Thus, 
pendent grounds, justify and support this app 
confidence in its general validity. 
Although ostensibly concerned. only 
emotions commonly termed fear, anger, 
; ] that the reseca 
o the emotion of 
a four-fold classificatory system. 


with “the three unpleasant 
and sorrow,” these investi- 
gators nevertheless. discoverec arch methods. they em- 
ploved also. directed attention t hope, thus provid- 


ing, once again (see Section VHI), : 
evident, the emotions thus identified are 


not exactly the same as those previously stressed in this chapter; but 
ear and hope are, of course, 


ained. F 

and the writers cited would pre- 
of relief. The important 
and sorrow in their sys- 
The latter we have de- 


However, as is immediately 


the discrepancies are easily expl 
clearly common to both ‘systems; 
on to the emotion 
presence of anger 
ont in our OW n. 
situation where a promise (hope- 
and now we sec that such a 


sumably have no objecti 
difference hinges upon the 
fem, as opposed to disappointme 
occurs in : 
a withdrawn; 
of two implications: irretrievable loss, 
leading to. sorod and prov leading to anger. Thus we 
return to the original classification of fear, relief, hope, and disappoint- 
Ment, with the latter now bifurcated into sorrow and anger. As we 


Shall see in Chapter 11, this added 


portant, in accounting for the appe 
1 some cases and extinction in others. But for the 


c will be made to this distinction, Instead 
a more fundamental disagreement w ith 


fine i / 
ined as the emotion which 
arousi ; 

rousing stimulus) has beer 
Withdrawal may carry cither 


isional loss. 


distinction is operationally im- 
ance in the face of disappoint- 


Ment, of aggression it 
present no further reference 
We turn to what is perhaps 
the Hunt-Cole-Reis analysis- 
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These writers assume that frustration is the lar to the indere 
ing and identification of the negative emotions of anger, fear, and 


sorrow. Thev then sav: 


The cues for these three emotions have been formulated in two ways. 
In one formulation, upon which the first study to be reported was based, 
the cues were conceived to derive from the perceived timing of the frustra- 
tion. The notion of perceived timing served to describe on one continuum 
the nature of the relationship among person, obstacle, and anticipated goal- 
response. In one case, the obstacle may be evident without actually inter- 
fering with the goal-response, ic. it may be threatening. In the second 
case, the obstacle may be evident and blocking the behavior- sequence or 
course of action. In the third case, the blocking already has occurred, and 
the goal is lost irrevocably. The three cases were - described as future, present, 
and past perceived timing. The finding of the first study prompted a change 
in the formulation, and the reformulation was then tested in the second 
studv to be reported (p. 140). 

Discrimination of fear, sorrow, and anger would, in terms of the rev 
formulation, depend in general upon S's perception of his relationship in 
a Digne situation to the goal being sought and to the presence of an 
obstacle. Fear should be named when S perceives a threat to his goal and 
uncertainty of achieving it. Sorrow should be named when S perc s 
the goal as irretrievably ‘lost, and when his concern is limited to the loss of 
the goal. For either of these two emotions, it is important that the concern 
with the frustrating agent or obstacle be minimized. Anger should be re- 
ported when this frustrating agent or obstacle is of central concern to S. 


(p. 146). 


Would it not be preferable to make threat, rather than frustration, 


the common basis for these emotions? And to assume that they are all 
future-oriented? Fear is thus identifiable as the prospect of the situa- 
tion getting worse (drive increment); ager as the prospect that some- 
thing hoped-for may be lost; and sorrow as the prospect that some- 
thing hoped-for is definitely lost. Hope, by contrast, is the prospect 
of the situation getting better (drive decrement), although there is 
always a possibility that it will not, which presumably explains why 
hope and fear, being mirror images of the same phenomenon, are 
dynamically always so intertwined. In this connection, Hunt, Cole, and 
Reis say: 


Situations intended to evoke fear brought out a new fact, namely, a re- 
lationship between hope and fear which was poorly recognized by the 
judges or sorters. For instance, only 5 of the 20 sorters put hope into the 
fear-category, while 2 put it in the anger. -category as a first choice and 6 
as a second choice. Hope was even put into the sorro category once as 
a first choice and once as a second choice by sorters. A much closer rela- 
tionship between hope and fear is suggested by the fact that they were the 
two predominant names for the emotions evoked by the situations intended 
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to arouse fear, Hope and fear also were given in response to seven of the 
equivocal situations, but these situations also clearly contain threats to the 
goal along with other cues. On the other hand, the Pope-response came 
only three times from the situations intended to evoke anger and never 
from the situations intended to evoke sorrow. It would thus appear that 
fear and hope are alternative responses to a situation in which the goal is 
thr ened.!* This suggests that w hich term is used predominantly may be 
a function of the degree to which S is inclined toward pessimism or op- 
Umism, If S is pessimistica ly inclined, the predominant response to be ex- 
pected may be fear, but if he is optimistically inclined, the predominant 
response to be expected may be pope. This finding and these considerations 
Suggest thar a battery of paragraphs like our descriptions of intended-fear 
situations might possibly serv ; a useful projective device to assess the 
degree to which individuals are inclined toward pessimism or optimism 


(Pp. 1497150). 


f we think about the emotions 


P 
s. Losses are, roughly, of 


erhaps the picture will be clarified i 
gains and loss 
shock) or a rise in secondary 


in the context of prospect 
two kinds: a rise in primary drive (c.g. 
drive (c.g., fear). And, in ‘like manner, gains may take cither of two 
i or a drop in secondary drive. A stimulus 
and a stimulus which sig- 


forms; a drop in primary drive 
Which signals a loss is said to arouse fear; 
1 hope. When a hope signal appears and 
disappointment and, more 
r or sorrow. Properly 


nals a gain is said to arouse 
then disappears, the reaction 
specifically, may take the form o 
Speaking, relief should perhaps not be termed an "emotion"; but if 
our earlier analysis in this chapter is correct. this is the essence of hope 
(type as well as of the experience which 

secondary reinforce- 


is one of 
f cither ange! 


2 secondary reinforcement) 
occurs when a danger signal disappears (ty per! 
Ment), 

Especially important, 
develop a relatively. inclusive 


here, is the fact that it now scems possible to 
“theory of the emotions” around the 
primary drive t 
(cf. Chapter 6) and that the four main 
relief, hope. and disappointment—can 
or phase of this basic state, situationally 


basic notion that every strong is accompanied by an un- 


differentiated “arousal state" 


emotions thus identificd— 
all be interpreted as some aspect 
defined, 

here are, obviously. 
DOW is known that we can ney 


mediately at hand. Certain of the problen 
be considered again. in different. contexts, 1n Chapters 6 and rr. 


still unsolved problems in this area, but enough 
-ertheless proceed with the tasks im- 
ys raised in this chapter will 


"astute psychological insight," 


as saying, W ich 
or hope without fear” (p. 6). 


lav (1950) quotes Spino 


Fear cannot be without hope. n 


Secondary Reinforcement: 
Reservations and Complications 


Because the theory of secondary reinforcement which has been 
presented in the preceding chapter is so central to much of the argu- 
ment that is to follow, it will be useful, at this juncture, to examine 
some of the criticisms which have been directed against this approach 
and to determine in what Ways it is most questionable or incomplete. 


I. Harlow’s Criticisms of the 
"Homeostatic" Model 


Perhaps the most scintillating skepticism with respect to the fear- 
reduction interpretation of secondary reinforcement has been cx- 
pressed, in a series of experimental reports and theoretical papers, by 
H. F. Harlow. Fortunately, for our purposes, these have recently been 
admirably set forth in a compact, highly readable paper (Harlow, 
1954) \ hich will form the basis of the present discussion. 

After some preliminary remarks, Harlow comes to the heart of his 


thesis, thus: 


Motivational theory and research have in the past put undue emphas 
upon the role of the homeostatic driv: es—hunger, thirst, sex, and elimination 
—as forces energizing and directing human behavior. There are, indeed, some 
psy chological theorists who would have us believe that all or most of our 
adult human motives are either directly dependent upon these homeostatic 
drives, or are second- or third-order derived drives conditioned. upon 
visceral needs. The fact that derived drives based on homeostatic needs 
are unstable and transient, the fact that the conditioned drive stimulus does 
not apparently reinstate the unlearned drive state (Miles & Wickens, 1953; 
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joes Wickens, Brown, & Pennock, 1951) 
carn and live for davs, weeks, or months without or in spite of a particular 


homeostatic need state do not disturb such psychological theorists in the 
least. It is, of course, the privilege of theorists to look at man from the point 
of view of the pylorus or to look at the pylorus from the point of view of 
man, y 

Recently, some psychological theorists, including Mowrer (19503) and 
Brow n (1953), have emphasized another area of motivation—pain and con- 
ditioned pain or anxiety and some psy chological theorists have argued that 
these motivational mechanisms could directly or indirectly underlie much 
human learning and could be extremely powerful forces in shaping human 
personality structure. No ps -chologist will underestimate the importance 
of conditioned pain (or wiety, if it is so defined), because this derived 
drive appears to be far less susceptible to experiment extinction than are 
the derived drives based on the visceral need states. But lest the role of 
conditioned pain and fcar be overestimated, it should be pointed out that 
common sense tells us that the greater part of our energies arc motivated 
by positive goals, not escape from fear and threat (pp. 37738). 


. and the fact that human beings 


In what has just been said it is implicd that the secondary drives 
derived from primary drives other than pain are qualitatively and 
usually also quantitatively. different from fear. In the preceding chap- 

“on the opposite assumption, apparently sup- 
ported by a good deal of empirical evidence, that every drive, of con- 
siderable intensity. arouses some degree of fear and that it is this 
reaction which is most readily conditioned to formerly neutral stimuli 
that precede sudden drive onset or accompany drive of mounting 


intensity, Hence the supposition is that all secondary drives acquired 
the same: they are presumably all species 
Somewhat later in this chapter, this generaliza- 
fied (Section V); but the point is, this ap- 
alls “positive goals” and “escape from 


ter, we have proceeded on 


in this way are, basically, 
of the fear response. 
tion will be slightly quali 
proach integrates w hat Ha 
fear and threat" into a uni 


rlow € 
fied svstem. 
s of the experimental findings reported by 
we now know that Harlow over- 
nature of “derived drives 


n F urthermore, on the bas 
Zimmerman and others (Chapter 5). n 
emphasized the “unstable and transient nasi ; 
based on homeostatic needs.” However, In light. of much of the 


previous evidence, this misinterpr 
against the homeost 


eration was a very natural onc. 


Harlow^s argument atic view continues: 

Furthermore, we would emphasize that intense emotional states are the- 
oretically unsatisfactory motives for many learned activities of a moderate 
or high degree of complexity- It has bee recognized for subhuman animals 
from the time of the formulation of the Y erkes-Dodson law (1908) that an 
Inverse relationship exists berween the intensity of motive and the complexity 
of task that can be efficients learned (p. 38)- 
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In the preceding chapters our assumption has been that many in- 
stances of secondary reinforcement are provided by a reduction or 
termination of relatively wild fears or fear-like tensions, and we have 


not at all assumed that “intense emotional states" are in any wa) 
prerequisite to the occurrence of this phenomenon. Diminutions in 
the level of intense and less intense secondary drives are assumed to 
be qualitatively similar and to differ, in their capacity to provide a 
reinforcing state of affairs, only in a quantitative sense. It is well 
known, of course (sec, for example, Young, 19492; ‘Taylor & Spence, 
1952; Montague, 1953; Spence & Farber, 1953; and Gordon & Berlyne, 
1954) that intense fears or anxieties can have a markedly disorganizing 
effect upon problem-solving behavior, presumably: because, when in- 
tense, they tend to distract the subject from the problem at hand and 
make him respond to and attempt to resolve the anxiety itself. But 
short of this threshold of distraction and disorganization, fear states 
seem quite capable of acting as suggested (cf. Chapter 5). 

But the foregoing is a comparatively minor consideration, and we 
return again to Dr. Harlow^s presentation. 


It is certainly not our desire to underestimate the importance of either 
eral drive states or emotional conditions as motivating forces under- 
lying learning and influencing personality. formation. Visceral drive states 
are important motivating mechanisms in children, and they become im- 
portant motivational mechanisms in human adults under deprivation. Further- 
more, the appetitive mechanisms [sec pp. 196 ff.], innate and acquired, 
which are associated with (even though not of necessity derived from) 
the visceral need states are important and persistent human motivational 
mechanisms. Emotional conditions including pain, fear, anger, and frustra- 
tion are also important and persistent human motivational mechanisms. 

But above and bevond the visceral need-appetitive motivational mechanisms 
and the emotional motivational mechanisms there is, we believe, a third 
category of motives, a category of motives which are clicited by external 
stimuli and which have been described by such names as manipulation, ex- 
ploration, curiosity, and play (p. 38). 


Lest Harlow’s “third category” appear more distinctive than it 
actually is, let us recall some of the assumptions made in the preced- 
ing chapters. In. Chapters 2 and 3 it is posited that fear (secondary 
motivation) can be conditioned either to response-produced stimuli 
(as in punishment) or to independent stimuli (as in avoidance learn- 
ing); and in Chapters 4 and ; it is posited that fear reduction (scc- 
ondary reinforcement) can, likewise, be conditioned cither to inde- 
pendent or to response-produced stimuli (sec also Chapter 7). Hence, 
if the third-category motives mentioned by Harlow are cued off by 
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external stimuli, this fact alone does not set them apart from other 
secondary motivators and reinforcers. Moreover, it will be noted that 
the descriptive terms used to identify this third-order drive—“manipu- 
lay" —refer to forms of behavior. 
it, the identification of drives in 
the reason that a given 
r and the same be- 


lation, exploration, curiosity, and pla 
Vhough there is much precedent for 
terms of behavior patterns is precarious, for 
erent types of behavio 
of different kinds of drives. For 
it is perfectly possible to have 


drive can produce many diff 
havior may be produced by a variety 
example, to illustrate the latter. point, 
two rats, both of which will leap into the air when placed in a demon- 


stration apparatus of the type described in Chapter 2; but one of 
them will be doing so because it is afraid of being shocked and the 
1 "begging" for food. It would 


other ines 
ther because it is hungry, as a means of 
ior of these two animals, to 


infer the presence of a "jumposity " drive. Various earlier writers (¢.2., 
Murray, 1938) have explored the possibility of naming drives after 
characteristically recurrent actions OF action patterns, somewhat in 
the Way, at a still carlier time. psychologists used to spend a good deal 
ob their energy—not very fruitfully. it now appears—drawing up “lists 
or instincts." A more dynamic approach to behavior theory seems to 
require that motives and the motions they produce be carefully dif- 
crentiated (sce Brown, 1953. PP- 11-185 Farber, 1954 pp. 24-25 
Miller, 19sib, p. 467): Although there is a certain limited sense in 

: “habit” become almost synonomous (Chapters 
results from inferring specific drives from 
re independent criterion! 


be most unwise, on the basis of the behav 


NS Sue . " 
Vhich motivation and 


& ae 
i and 12), only confusior 
chavior rather than identifying them by son 


1 H “ " H ' 4 enti 1 imi itv erw eel 
" It will perhaps be helpful at this point to call attention to à similarity between 
le position which Harlow is here advocating and th 


sive F : A : Aet 
© Fdueation. Under the leadership of John Dewey. thc 
and coercion thar cha acterized many 19th-century 


the importance of the child's ow "needs and in- 
"negative," the newer one, “positive.” 
the positive," any recourse to 


e general stance of Progres- 
the latter movement was in 


rebelli " 5 
al Pelion. against the fear 
ASST H 
srooms and stressed instead 
Crests.” as said to be 
“accent 


Fhe olde 


pproach w 


And since it is currently fashionable to c | a ; t 
Negative” methods is regarded. as somewhat immoral. Two considerations are in 


Order here. Ta the svsrenatie conception of motivation and reinforcement which 
hay Been alelincared in. Chapter 34 WV have made theoretical provision. for both 
types of motivation and learning— GU motivation aroused by a danger signal and 
Secondary. reinforcement. (type-1? provided by the reduction thereof and (b) the 
Motivation (fear interest) aroused by seme already present drive or need and 
the secondary reinforcement. (type?) v hich a safety signal or promise produces. 
What the Progressive Educators, in effect, 


ir à ini snerally 
" the classroom (and in child ining generally) í Y 
tions, We do not in the least claim that the two situations are identical, or that 


do is to stress the greater desirability 
of the second of these situa- 
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Il. “Curiosity” and Fear 


It is well known that the concept of imitation was weakened by 
the circular way in which it was commonly used, i.e., as both a descrip- 
tion and as an PEDIR of behavior ( Mowrer, 1960, Chapter 3). 
And it now appears that "curiosity" suffers from the same abusc: it 
is either a phenomenon which is to be explained, or it is itsclf a force 
or drive that can produce certain specific effects, but it is not both. 
Said otherwise, it is not simultaneously cause and effect, The position 
we shall here adopt is that "curiosity" is an effect, which may take 
the form of either physical or visual exploration of the environment 
(or both), and that fear is a common cause thereof. 

Speaking for what he believes to be a different and more legitimate 
conception of curiosity, Harlow (1954) says: 


The denial of the existence or importance of the externally elicited mo- 
tives is amazing because at the common-sense level of humor and aphorism 
there are many references to the operation of the external-drive mechanisms. 
It is recognized that all primates, including man, spend a large amount of 
time just “monkey ing around" and that monkeying around is an activity 
often leading to invention and creati ity. There are ‘countless cartoons bear- 
ing on the theme that the monkeys in the cage stare at the people outside 


E 


hey do not have differential consequences. For example, a teacher (or any other 
person) who is constantly emitting danger signals will tend to drive students away 
from her (and from the school situation in general), whereas a teacher w ho emits 
safety signals and promises (which are confirmed) tends to attract them. And 
this is important, educationally. But our approach makes provision for both forms 
of "motivation and learning" and shows that, in the final analysis, they are not so 
very different, theoretically. Such practical differences as there are hinge mainly 
upon the question of what arouses the fear (or “interest a danger signal from 
the environment or a primary drive (or "danger") within the individual him- 
self. The by- products of the two methods or situations are often very different 
indeed; but there i is, it seems, no great difference in theory or scientific principle. 
Moreover, it now appears that even the practical advantages of the “progressive 
approach are not so great as carly enthusiasts believed, in that some children are 
so secure and “well-fed” that there are no secondary motives or interests on which 
the would-be educator can build; and, in addition, there are some things which 
human beings have to learn which apparently have to come from external, “im- 
posed" authority and cannot be handled entirely on the basis of "indigenous? or 
"intrinsic" motivation. This line of thought cannot be further pursued here (ef. 
Mowrer, 1960, Chapter 10). But it may be useful to the reader, in evaluating the 
following discussion, to keep in mind that the issues which are here under scien- 
tific scrutiny have broad and, very often, emotionally loaded "philosophica al” 
implications. It is hoped that our analysis will reduce the emotion and increase 
mutual understanding. 
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and are just as amused by what they see as are the people. Kohler reported 
staring through a peephole to see What a chimp was doing and found that 
j was staring at him! Visual exploration drives in subhuman primates are 
clearly recognized by the saving “Monkey see, monkey do.” Yet, in spite 
of the oby ious existence of the external drives psycholog have per: isted 
n limiting themselves to endlessly repeating with insignificant variations, 
experiments designed to show the allegedly overwhelming importance of 
the homeostatic, internal drives. > + - 

Every comparative psychologist who has adapted rats for maze experi- 
ments knows that the rodents frequently run down the straight-away path 
ignore the food, and continue to explore 
“rat will refuse to cat until exploration 
ay have been deprived of food for 23 


used in the adaption procedure, 
the environment. Frequently. the 
and curiosity are sated, although it m 
hours previously (p. 39)- à 


But two other psychologists have previously interpreted similar 


observations quite differently. They say: 
Preliminary observations had indicated that if animals [rats] were placed 
on [an elevated] maze without prior habituation, they showed considerable 
anxiety, This was first indicated by great cautiousness of movement and ex- 
cessive urination and defecatio Later there was a period of feverish ex- 
ploration, during which the animals ignored food even though they had 
not eaten for 24-36 hours. These observations suggest that “curiosity” is 
perhaps more closely related to anxicty than is ordinarily supposed (Whiting 


& Mowrer, 1943. p. 230) 


an unfamiliar maze may, 


at hungry rats in 
at they are safe before 


Here the assumption is th 
make sure th 


quite understandably, want to 
they attempt to sate their hunger. Only after they are “satisfied” that 
there is nothing dangerous in the new situation are they able, it seems, 
: É nceds. And although the fear is here, 
al to the organism, there is no 
al than is the metabolic 


to respond to their nutritional 
to be sure, elicited by stimulation extern 
reason for assuming that it is any the less intern 
drive of hunger. Yet some writers show a persistent tendency to re- 
Ject fear as the cause of “curiosity.” although the facts are completely 
interpretation. For example, Montgomery 


ed experimental paper as follows: 


congruent with such an 
(1955) begins a much-cit 
t investigation is to determine whether novel 
stimulation evokes the fear drive as well as the exploratory drive. A growing 

ates that novel stimuli evoke the exploratory drive 
Montgomery, 1951b, 19533 and Thompson & 
ation has rep atedly suggested to the author 
the fear drive. In the psychological and 
has been made a number of times, 
supporting it can be found (e.g. 
and Rand, 1941) (p. 254)- 


The purpose of the presen 


body of evidence indic 
(Butler, 1953; Harlow, 19505 
Heron, 1954). Qualitative observ 
that such stimuli may alse evoke 
Zoological literature the same suggestion 
and scattered and unsvstematic €V idence 
Hebb, 1946, McDougall, 19235: Melzack, 195 
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What this writer neglects is da possibility. that n and the "ex- 
ploratory drive" are one and the same thing. This oversight appar- 
ently occurs because of the assumption "that the exploratory drive 


leads to approach behavior (Berlyne, 1950; Montgomery & Monk- 
man, 1955) and that the fear drive leads to avoidance behavior (Miller, 
1951b; Montgomery & Monkman, 1955)" (p. 254). The latter proposi- 
tion is clearly gratuitous. It is true (as already scen in Chapter 2) that 
if a source of discomfort has been clearly and definitely localized in 
space, the resulting fear leads to avoidance; but it is also well estab- 
lished that a vaguer, less-well- identified threat can cause vigorous 
generalized activity and even specific tendencies to return to the place 
where some painful experience (such as electric shock) has occurred. 
Rats shocked in one of two adjacent compartments and thereby driven 
into the other show an almost irresistible tendency to go back to the 
first compartment (cf. Chapter 2). And Hebb & ‘Thompson (1954) 
report that: 

. the dog that is frightened by a strange object is nevertheless apt to 
return to look at it again, balanced between closer approach and flight 
(Melzack, 1954). The same thing can be observed in chimpanzces, and 
Woodworth (1921) and V alentine (1930) have described the behavior of 


young children who ask to be show n again—at a safe distance—the object 
that has frightened them (p. 551).? 


The biological utility of such a tendency is clear: it is important 
for living organisms to know whether a given event is accidental or 
is associated in some orderly and predictable way with a particular 
puse or action, Therefore, a certain amount of "exploration". (or 
“reality testing’ `) is necessary and useful as a means of more definitely 
“structuring” a situation, i.c., frecing the individual from what might 
otherwise be needless inhibitions (or “superstitions” ) or confirming a 
specific and reliable source of danger. And fear—diffuse, nonspecific 
fear—can serve as an adequate motivation for such behavior; there is 
no need to posit a special (and tautological) * 'exploratory" or "curios- 
itv" drive. 

This tendency for organisms to re-expose themselves to rather non- 
specific dangers has suggested to some the possibility that the “ego likes 


*On the basis of extensive experience in breeding and training horses, Miss 
Gertrude Hendricks reports that “Young colts are afraid of everything that is 
strange. But breeders and trainers know that colts react in two different ways 
to strange situations: some of them just avoid them—others investigate them. Only 
the latter are regarded as good candidates for training" (personal communica- 
tion). Cf. Welker (1956), “Effects of Age and Experience on Play and Exploration 
of Young Chimpanzees.” 
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to precipitate anxiety in manageable amounts" and that this tendency is 
the basis for masochism. However, such speculations have not received 
sen and Zimmerman, at the 


laboratory confirmation. For example, E 
University of Illinois, taught rats to press a bar as a means of turning 
off a danger signal and averting an electric shock. Then they made it 
possible for the subjects, by means of another manipulation, to turn 
on the danger signal, which they could, of course, immediately termi- 
ar. Suffice it to say that the rats showed no 


nate by means of the b 
ager signal on, because it “felt so good" to 


inclination to turn the dar 
turn it off. Apparently, at this point, it was perfectly clear to the rats 
that the danger signal was something they wanted as little of as pos- 
c " try to “master” (extinguish) their 
fear of it or to "explore" it. Here, it scems, the situation was already 
too highly structured (perceptually) for the rats to have any "curios- 
itv" about it whatever: they definitely knew that the danger signal 
Was bad and that the less they had of it the better. Only, it seems, 
in situations which are in some way ambiguous do exploration and 
curiosity manifest themselves. Whiteis (1955, 1956) has shown that 
rats will continue to give themselves a brief shock as a means of escap- 
" but this phenomenon has been rather thor- 
19502) and has no special 


sible; and there was no tendency to 


ing from a fear situation; 
oughly analyzed in other terms (Mowrer, 


relevance here (see Sidman & Boren, 1957b; also Chapter 12). 


lll. The Problem of Visual Exploration 

vation reported by Harlow? This 
involves putting a monkey Il light-proof box and rewarding 
T or discrimination behavior, merely by a brief 
6-1; also Butler, 1953). Of the results 
a summary of his general point 


But what of another type of observ 
< into a sma 


It, for manipulative 
"look" out of the box (see Fig. 
of this kind of experimentation and as 
of view, Professor Harlow says: 


We are convinced that the externally. elicited motivational systems are 
as fundamental and as innate as are the hunger-appetite and thirst-appetite 
systems. By this we do not mean that learning does not operate as à com- 
ponent. in the externally elicited motivational systems of the adult animal: 
We merely mean that the learning component here is probably no larger than 
it is for the hunger-appetite motivational system. We firmly believe that 
externally. elicited motivational systems interact with the hunger-appetite 


and thirst-appetite systems. but we do not believe, and there is no evidence 
H at any of these systems 15 derived from the other, 


to support the position, th 
or that any differential de 


gree of dependence exists among them. 
Recently, motivation theorists 


and personality theorists have shown some 
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motivational obsession about anxiety. No one will deny that anxiety, or 
many other emotional states, serve as motiv but any assumption that 
anxiety has some special, prepotent motivational role has vet to be estab- 
lished [ cf. Chapter 5, footnote 15]. At best, anxiety is a motive for avoidant 
behavior |cf. Section II], and the greatest part of human motivation is 
positive scarching toward goals, not mere avoidance. In spite of our ith 
in the importance of positive, forw ard-oriented motives such as curiosity, 
manipulation, and exploration, we do not wish to put any constraints, in 
research or theory, upon the anxious psychologis ; We merely w ish to 
insist, however, that even if some psychologists ar scourged to their ex- 
perimental dungeons like quarry slaves, the remainder of the population 
will continue to be motivated bv pleasant and positive incentives (1954. p» 52)- 


Fig. 6-1. Sketch indicating a portion of the research set-up dev cloped by Harlow 
(1954) for the study of visual motivation and reward in the monkey. When the 
door is closed, the subject readily learns to solve puzzles and make complex dis- 
criminations as a means of getting the door to open for a few minutes. The op- 
portunity to "look out" is the only reinforcement received for such activities. A 
theoretical analysis of this type of learning is suggested in the text. 


Harlow is, of course, right in saving that many psychologists have 
stressed the “negative 


side of motivation and neglected the "positive 
side. As indicated toward the end of Chapter 2, there is, however, à 


historical, if not logical, reason for this; and the two immediately 
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preceding chapters show the extent to which psychologists have, more 
recently, concerned themselves with the “pull” ather than the “push” 
side of the motivation and learning problem. Nevertheless, one won- 
ders if it is not a contradiction in terms to speak of "pleasant and 
any years ago. in a very thought- 
1936) took the position that all 
“negative,” in that it involves a 


positive" motives or drives. A good m 
ful paper, Muenzinger & Fletcher ( 
motivation is, almost bv definition, 

state of affairs which the organism is stri 
nate. In order to reduce hunger, for example, the organism must, to be 
sure, behave “positively” with respect tc 
in its environment; but this “adient” behavior, to borrow E. B. Holt’s 
terms, is basically “abient.” In other words, the organism tries to get 
: as a means of having less of certain 
c of reference, hunger is the motive 
aders if Harlow’s discussion is not 
asv, seemingly uncritical use of 


iving to alter, correct, termi- 


» certain substances or objects 


more of certain stimuli (food) 
other stimuli (hunger). In this fram 
and food is the incentive, and one wor 


misleading in the wav it involves an € 
one as virtually synonymous with the other (sec also Chapter 8). 
But perha s all this can be made more specific by looking carefully 


at the methodology which Harlow, in the paper under discussion, 
E procedure. Harlow has steadily 


little cell-like box for a 
as involving reward or 


Stresses most: the work-for-look 
argued that the privilege of peering out of the 
few minutes cannot reasonably be thought of \ 
reinforeement in the sense of ‘any kind of drive reduction. Pretty ob- 
Viously, there is no primary drive gratification, such as hunger or 
c has equally held, with less clear justification, 
no reduction in any sort of secondary drive, 
Congress of Psychology, held in 
"s a presentation. by 


thirst reduction; but h 
that there is likewise 
Such as fear. At the International 
Montreal in 1954, N- E. Miller, in response to res 

i “solitary confinement 15 one of the 


Harlow, pertinently observed that à j 
most dreaded methods of disciplining human prisoners; and, being only 
a little less social, the monkey probably finds it equally odiov If 


Myers & Miller (1954) as follows: “We 


rated by pha j 
restraint 15 frustrating, or 


ement produces 3 axiety, ; 
], the observation of small children 


Sere A 
This point has been elabo 


believe ic possible that confin 


monotony arouses a drive of boredom. Indeed rvati¢ 
i i ill, the reports of prisoners subjected to 


absolutely st 
difficulty 
he effects of decre 
strong motivation [see also Lilly, 


who are required to sit 
Solitary confinement, and the 
retaining Ss in their experiment on t 
Would indicate that such conditions can produce 


1956; Heron, Doane, & Scott. 19561- Therefore, W 2 3 
ontonous stimulation, enforced inaction, etc. may be re- 
freedom of action, Cte and that such drive reduction 
ed in learning. for ‘exploratory, ‘manipulatory, and 


of Bexton, Heron, & Scott (1954) in 
ed sensory variation 


c suggest that drive produced 


b 

i homogeneous or m 

duce : 

duced by sensory variety. 

Is 3E 257 ij ee; 
the reinforcement involv 
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visual isolation is objectionable, then even a brief period of visual 
freedom would be rewarding and could be used to facilitate, perhaps 
quite powerfully, learning of various kinds (see Miller, 1954). 

It has long been known among bird trainers that one of the best 
Ways to motivate a bird to “talk” is to cover its cage so that it cannot 
see “what is going on.” Having thus been deprived of visual contact 
with its environment, the bird tends to concentrate upon the oral- 
aural channel of communication and will listen more attentively to 
the voice of the trainer and will, itself, be more likely to vocalize. 

Even more dramatic and more recent evidence comes from the 
psychology laboratories at McGill University (Bexton, Huron, & 
Scott, 1954) of the disturbing effects produced upon human beings 
by cutting off, as nearly as possible, all sensory contact with their 
environment. In a dark, silent room, with even cutancous and tactile 
stimulation greatly reduced, human subjects, in the course of three 
aving experiences which, under 
nly be regarded as psychotic. 


or four days, often find themselves h 
ordinary circumstances, would cer 
These experiences, fortunately, are transitory and lift with a return 
to normal experience; but they show how disagreeable and disorganiz- 
ing sensory deprivation can be. The box employed in the. Harlow 
experiments is presumably a sort of miniature version of the isolation 
room used in the McGill experiments; and one would predict that 
monkeys would not only learn to do whatever would temporarily 
end the visual isolation, but would also show great alacrity in leaving 
such a box, when given the opportunity, and also reluctance to re- 


enter it. 

Nor is it difficult to see why exclusion of visual stimulation should 
make for apprehensiveness. In terms of orientation and movement in 
space, vision is by all odds our most valuable sense; and one can quickly 
few minutes to 


gain a nice appreciation of it by attempting for 
carry on, blindfolded, his routine activities, Evervone has vivid 
memories of falls and assorted bumps and bruises received when mov- 
ing about in darkness; and vision is perhaps even more useful in the 
social, interorganismic world. Being able to see, to “lay eyes" on, 


‘exercise’ rewards. For the present, we choose the more simonious alterr 
tive of trying to analyze such concepts as exploratory drive along the same lines 
as conventional drives” (p. 435). Cf. Miles (1958b) who, in a paper on learning 
Min kittens with manipulatory, exploratory, and food incentives," says: “The 
results support the thesis that manipulatory and exploratory activities are reward- 
ing in their own right and that the postulation of a derived-incentive function 


is unnecessary” (p. 41) 
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logic mr seem to require that in Harlow’s experiments the drive 
be in the monkey, in the box, before the little window is opened and 
that what is then seen, on the outside, is rewarding, not “motivating” 
(except in so far as a reward always sets up an incentive system or 
"habit"—see Chapter 7). 

Dr. Harlow has certainly reported some interesting and important 
observations; and he has, in addition, performed a valuable service 
in alerting us against the danger of a too ready acceptance of the 


"homeostatic" conception of motivation and reinforcement. However, 
it can scarcely be said that any of his experimental data constitute 
crucial, or even very strongly presumptive, evidence against such a 
view. In fact, an examination of these experiments, at a little deeper 
level of analysis, seems to provide support for the very position which 
Harlow so vigorously (though charmingly ) rejects. 


Illumination as Reinforcement and the 
Concept of Optimal Stimulation 


Closely related to Harlow’s researches is a study by Kish (1955) 
on the use of a moderate increase in illumination as a source of rein- 
forcement. This study was suggested by another (Kish & Antonitis, 
1956), in which it was found that “a class of events that appears 
neither to be need-reducing nor previously associated with need re- 
duction [such as] microswitch clicks, relay noises, and stimuli pro- 
duced by a moving platform exerted what appeared to be reinforcing 
effects on the unconditioned operant behavior of mice” (p. 261). 
These investigators were disposed to think of these events as € 
forcers," and in a certain spec il sensc the - were: they reinforced, ¢ 
strengthened, general exploratory activity, but they seem to ed 
done this by continually introducing novel stimuli into the environ- 
ment, thus keeping the mice "interested," aroused, stimulated, curious. 
apprehensive, motivated (cf. Walker, Dember, Earl, & Karoly, 1955): 
There is not, so far as one can see, any indication here that the mice 
were reinforced in the sense of being rewarded or relieved by these 
stimuli: quite the reverse. But the experiment to which these ob- 
servations led was of a different order: it involved putting laboratory 
mice into a dark box and then making it possible for them, by means 
of a “bar-touch response, ` to turn on “dim illumination." After seven 
daily habituation sessions (of 25 minutes each) in the box, the subjects. 
on the eighth day, found that the bar, which had been previously 
available but * "dead." now caused the box to be briefly illuminated: the 
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ssion, now increased from 


number of contacts with the bar, per 
about 70 to 160. And on five subsequent "extinction" days the inci- 
ably higher than it had been in the 


ndicating that this response had 
on the one day it had been rein- 


dence of bar pressing was consider: 
preceding habituation sessions, thus it 
acquired considerable “habit strength” 
forced (i.c., had turned on the light). 


Kish is therefore obviously on sound grounds when he savs: 


that the onset of dim illumination occurring 
leads to behavioral effects which corre- 
a for the occurrence of reinforcement 


The data clearly demonstrate 
as the consequence of the response, 
spond to those noted as the criteri 
(p. 263). 


But the justification is less good when he goes on to say: 


ar to support the hypothes s that a per- 
which is unrelated. to such need states as 
any response which it follows (p. 263). 
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if you are “looking for something," either good or bad, ability to see 
it is of the essence (see also Mowrer, 1960, Chapter 5).? 

That the value of light to laboratory animals under the conditions 
just described is that it enables them to see, rather than providing 
some inherent "primary reinforcement" of the kind posited by Kish, is 
clearly suggested by observations recently reported. by Hurwitz 
(1956b). With rats as subjects, this investigator says: 


When the light is kept on [by means of bar pressing | for extended periods, 
the animal vigorously surveys the box, making rapid, scanning-like head 
movements. The rate of pressing was not regular. Responses tended to be 
bunched, each “burst” being separated by periods of "silence." During à 
response burst the animal would stand on its paws on the lever, the head 
held at right angles pointing to the light source (p. 33). 


Since it is thus so apparent that the rats were here using the light 
as a means of scrutinizing the environment, it is somewhat surprising 
that Hurwitz goes on to comment, thus: 


The above results are relevant to the controversy about the importance of 
homeostatic mechanisms in animal learning which continues to feature 
prominently in the literature on the psychology of learning. Some writers 
(Hull, 1943) hold that the conditioning of respons s is only possible in the 
presence of stimuli representative of need stabilization. So far, at least, it 
has been extraordinarily difficult to adduce evidence inimical to this theory. 
However, since it is not obvious how light—in the case of rats—is instru- 
mental to the stabilization of organic needs, it seems that the experiments 
reported here, as well as those reported bv Girdner (1953), Henderson 
(1953) and Marx et al. (1955) call for an alternative approach to the central 
problem of conditioning (p. 33). 


Any paradox which may be apparent here is resolved by the as- 
sumption that every primary drive has an affective component and 
that the latter can be cither heightened or lowered by external condi- 
tions that are, respectively, unfavorable or favorable to the prospect 
of primary-drive reduction. Also, it is to be remembered that affect 
can be aroused in the absence of strong primary drive ( but in anticipa- 
tion thereof) and that it, too, is a potential source of secondary rein- 
forcement which can be mediated by environmental cues or circum- 
stances. ) 


*The possibility should be explored that familiarity with a situation will 
influence a rat's interest in illumination. If the size, shape, floor texture, and smell 
of a compartment were w cll known, a rat might immediately identify where he was 
and, especially with the aid of vibrissae, be able to get around in the situation 
quite w ell without light, hence show less tendency to perform an act which would 
turn light on than would be evident in a strange situation. 
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f Experiments by Keller (1942) and others have shown that intense 
light. (like clectric shock) can be used (a) to motivate and reinforce 
laboratory rats in the development of escape responses and (b) as 
the unconditioned stimulus for the development of anticipatory, avoid- 
ance reactions, Therefore, since the work of Kish and others shows 
that when in total darkness (and otherwise properly motivated) 
laboratory animals will also strive to ge? illumination, it is not surpris- 
ing that the concept of "optimal stimulation" should have occurred 
Leuba (1955). after reviewing the work 


to some writers. For example. 
ers, comes to the conclusion 


of Harlow, Montgomery, Kish, and oth 
that: j i 


Tension reduction would not seem to be a general or the only principle 
of learning. It may be only one aspect of a more general principle, a prin- 
ciple which might be called one of optimal stimulation: the organism tends 
to learn those reactions which will produce an optimal level of total stimu- 
lation. Hebb writes “animals will always act so as to produce an optimal 
level of excitation” (Hebb & Thompson, 1954. P: 552)+ That level is often 
higher than the one present at the moment (p- 28). 

Both the experimental evidence now àv able and general observations 
seem to indicate that the concept of optimal stimulation may have wide 
applicability in human development. Tbe organism tends to acquire those 
reactions which, when over-all stimulation is low, are accompanied by in- 
creasing stimulation; and * ben over-all stimulation is high, those which are 
accompanied by decreasing stimulation (p. 29)- 

a point of view that can be summarily dis- 
to be uncritically accepted. As 


We have seen, situations which m to violate the homeostatic 


Model often prove, upon further analysis, to conform to it very ac- 
Ceptably. Furthermore, the principle of optimal stimulation proves 
“cuts both ways” and can “explain” anything 
but is weak on prediction. Also it should be 
theorists would today in any case, hold that 
re only principle of learning.” As repeatedly 
also occurs in connection with drive 


This, certainly, is not 
missed; at the same time it is not one 
at first scc 


hard to test becausc it 
that happens, post liba. 
Temembered that few 
"tension reduction [is] tl 
noted in this volume, learning ; i i 
increase; but this learning is different from that associated with drive 
decrement. Therefore, many 
(and, at a superficial level, are) secl 
Sent attempts to reduce a more fundar 
1 on the basis of pr 


instances where organisms seem? to be 
king stimulation turn out to repre- 
nental tv pe of stimulation (fears 


or tensions) acquired ior learning of the incremental- 
reinforcement variety. 

Yet another consider 
cal principle of optim 


ation that is relevant here is that the physiologi- 
al state does not necessarily provide a valid 
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analogue of psychological functioning. Mammals, for example, must 
maintain a given bodily temperature, within rather narrow limits, or 
die; and when this state goes cither above or below optimum, steps 
are taken to correct it. However, from a psychological and motiva- 
tional standpoint, temperature deviations both above and below "nor- 
mal" function in similar ways; each makes the organism uncomfortable 
and so, basically, may be said to drive it in only one direction: toward 
reduced, rather than heightened, stimulation. Even though secking a 
higher temperature physiologically, what the organism is trving to 
do motivationally is to get rid of the noxious feeling of coldness, or 
hotness, as the case may be. 

Despite these negative indications, however, onc should not com- 
pletely exclude the concept of optimum stimulation or tension. Be- 
yond a certain point of relaxation or tensionlessness, living organisms 
do, after all, cease to function in the behavioral (psychological) sense 
and go to sleep (a purely physiological state). And waking is now 
commonly thought to involve an “arousal reaction” (Delafresnaye 
et al., 1954). Even a steam engine, an electric motor, or an internal- 
combustion engine must have a minimum of internal "strain," pres- 
sure, or torque to operate. And the McGill University studies (already 
cited) suggest that after human beings, in an environment of. ab- 
normally constant and low stimulation, have "caught up" on their 
sleep, they find such an environment strangely uncomfortable and 
disconcerting, The issue is therefore by no means a closed one; but 
it seems that there arc still unexploited possibilities in respect to 
drive-reduction (homeostatic) theory which should be carefully con- 
sidered before one resorts to the notion that drive induction can be 
rewarding (rather than punishing). 


V. Affectivity, Drive, and Need: 
The Views of P. T. Young 


Over a period of some years now the researches and writings of 
Professor P. T. Young have generated considerable interest in the 
gencral area of motivation and learning. Fortunately, these studies 
have recently been summarized in two papers. which will form the 
basis of the present discussion, The most salient conclusions rcached 
at the end of these two papers rcad, in part, as follows: 


Affective processes exist in the rat as truly as in man. When the head 
receptors, especially those of taste and touch and smell, come in contact 
with a food there is an affective arousal which we have designated as enjoy- 
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ment... Distress produced. by deprivation and the relief of distress 
through food ingestion are also affective processes which are importantly 
related to food acceptance. / 

An hedonic theory of drive is proposed. Specific food-secking deter- 
minations are organized within the nervous system to preserve the enjov- 
ment of foods and to relieve organic distress produced by dietary depr ; 
tion, In general, rats develop drives to run to foods which they like (find 
enjoyable) rather than to foods which they need (require nutritionally ). 
There is, however, a positive correlation between what rats like and what 
they need but not a one-to-one relation (Young, 1949b, p. 119). 

The sign (positive or negative), intensity, and duration of affective 
processes are of prime importance in the organization of behavior, espe- 
cially in the organization of approach and avoidance patterns. Current 
stimulus sponse psychology is completely inadequate as an explanation 
of the many facts considered in this paper. American psychologists, domi- 
nated by the SR principles, have followed Watson in rejecting affective 
and the inadequacy’ of introspective 


processes on grounds of subjectivism j 
methods. But a wholly objective approach to problems of affective arousal 


Is now possible and, moreover, such an approach is necessary. The S—R 
approach has marked limitations in the analysis of preferences, optimal ac- 
ceptance of foods, acquisition of attitudes and motives and related problems. 
his S—R approach must be supplemented by a theory that recognizes the 
central importance of affective processes in motivation and learning and 
that supplements S—R principles by sound hedonic theory (Young, 1955, 


PP- 232-233). 


ent more specifically on these excerpts 
helpful to provide some background 
ing as a psychologist at 


Before attempting to comm 
from Young's writings, it may be 
material. Young received his graduate train 
Cornell University, under the great Structuralist and introspectionist, 
V. B. Titchener. Remnants of phrase and concept from the Titchenerian 
yon in his writings; but, unlike most students w ho 
*s influence, Young went into animal research; 
common Watsonian orientation, he was able 
ict Behaviorist felt he must 


school are still comn 
came under Titchener 
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Young's writings are involved, terminologi- 
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of Young's reasoning with 
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cally. difficult, and somewh 
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the place where the shock was received, I would assume a negative affective 
process. In both situations the affective process is inferred as a cause or 
source of behavior and as a factor that sustains or terminates the pattern 
aroused, The acquisition of approach and avoidance patterns proceeds ac- 
cording to the hedonic principle of maximizing positive affectivity (de- 
light, enjoyment) and minimizing negative affectivity (distress). The hedonic 
processes thus regulate behavior in a very fundamental sense. They underlie 
the direction that is characteristic of many acquired motives (p. 194). 


In terms of revised two-factor learning theory, we would interpret 
Young's two illustrations in terms of the following principles: Re- 
sponses, stimuli, or situations which are associated with incremental 
reinforcement (punishment) take on the capacity, themselves, to arouse 
a form of discomfort (fear); and responses, stimuli, or situations as- 
sociated with decremental reinforcement (reward) take on the capac- 
ity to reduce discomfort (arouse hope); and, as a result, living or- 
ganisms tend to be guided away from punishing situations and toward 
rewarding ones. 

The point of difference, it seems, is this: Young takes the position 
that “a sip of sugar solution” is capable of acting as a “positive” rein- 
forcer without any necessary drive decrement. In support of this 
position he cites an experiment in which: 

Rats were fed ad libitum upon a standard diet and an unlimited supply 
of tap water. There was never any deprivation of food or water; no known 
dietary deficiency existed; there was no initial drive to be reduced. The 
rats were healthy and active throughout the experiment. Motivation came 


solely from contact with the sw ect-tasting fluid; the rats ran to get a taste 
just as a child runs to get a piece of candy (p. 196) i 


Young is here assuming that the actions and situation associated 
with the sip of sugar solution were positively reinforced. merely be- 
cause the solution tasted good, and not because it reduced any drive. 
This is the essence of his theory of “palatability” or “positive af- 
fectivity." First let us examine possible weaknesses in this interpreta- 
tion and then a supporting consideration, 

The passage just quoted contains some very bold and questionable 
assumptions. For example, it completely ignores the possibility of 
specific bungers. It assumes, quite unjustifiably, that because rats have 
no nutritional drives which water or some bulk food like Purina chow 
can satisfy, they have no nutritional drives at all. It is entirely possible, 
indeed probable. that certain types of hunger can be appeased while 
others remain unappeased (sec Young, 1955, p. 229). And this as- 
sumption is made all the more plausible by the fact of satiation: rats 
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ANTT M solution. This fact strongly suggests that some 

eda. pus ip rather than mere palatability, is here involved. 

eis Penes cited, the rats received only one sip per day, 

& Dutty ard nility of satiation was carefully avoided; see also Smith 
y. 1957). 

As pointed out in the preceding chapter, 
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fore provide an organism with an important advantage over one that 
had no such racially derived “knowledge.” 

What are the facts? There is a good deal of evidence that newly 
born mammals react differently, without prior learning, to sweet and 
bitter (Pratt, Nelson, & Sun, 1930). And certainly common sense sug- 
gests that we ordinarily taste things in order to scc if they are “fit 
to eat,” not that they taste good because they have, in the past, ap- 
peased our hunger. Yet it would obviously not be biologically uscful 


for our food likes and dislikes to be completely and irrevocably fixed 
by instinctive mechanisms." Laboratory facts seem to suggest that they 
are not. In the 1955 paper previously cited and discussed, Young 
reviews a number of experiments in which food preferences have 
been altered. It has been found that, as common experience would 
suggest, satiation on one distinctive type of food (say sugar) may re- 
verse the normal preference of that food for another (say protein). 
Such alternations in apparent palatability are, of course, transient. 
But chronic deprivation of animals with respect to a particular type 
of essential food may produce, as Young has shown, relatively en- 
during changes in the relative acceptability or palatability of ‘foods 
(see also Brown, 1955). 

In sum, then, what do we really know about the question of whether 


"These considerations can be given added cogency by thinking of taste, and 
smell too, as a kind of preliminary chemical analysis (or physiological "sam 
pling’) that is quickly made of substances as they enter the body, through 
the mouth or nose, These are the two normal intak portals of the body; and 
because it may be “too late" after a substance gets inside, it must be checked” 


before being allowed "admission." A less refined, second-stage checking also 
goes on, in the lungs and stomach, to be followed by ejection if the results 
are not good; but the greatest sensitivity and discernment is in the special receptors 
which are more peripherally located. Considering how much is here at stake, 
biologically, it is not surprising that there should be considerable “excitement” 
(“affective arousal,” in Young's terms), of cither a positive or negative kind, 
as the reports of such analyses flow to the central nervous system. (A good 
“poison,” obviously, is one which gets by both levels of censorship and is taken, 
from the stomach, or lungs, into the bloodstream.) For another dimension of this 
problem, see also Chapter 10. 

* Murray, Wells, Kohn, & Miller (1953) have recently shown that sodium 
sucaryl, a substance which tastes sweet to human beings, is rejected by rats. “This 
difference,” they say, “indicates that the taste mechanisms of the tw a species arc 
different” (p. 137). And they imply that the difference is inherent rather than 
experientially determined. “The affective value of taste stimuli is a little-explored 
area with important implications for a theory of taste (Pfaffman, 1951). Further- 
more, species differences in behavioral measures of taste receptivity may be a 
strategic point of departure for the analysis of the mechanism of taste” (p. 134)+ 
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r less completely, at birth? 


whether taste values are given, more o 
While it is apparently quite possible for an organism to learn to like 
or to dislike a particular kind of food (on the basis of specific ex- 
no doubt that some tastes are 


periences therewith), there can be 
cable or disagreeable; and Young has done a real service 
in keeping this fact from ‘being overlooked in an age of extreme (and 
quite possibly excessive) empiricism. However, it is still a mystery 
how the nativistic attitudes of acceptance or rejection toward partic- 
ular foods are achieved, neurologically speaking; and, as already indi- 
cated, these "affective" (palatability) reactions do not operate in com- 
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In a paper published in 1948 Young himself recognized this problem: 


We had previously assumed that the rate of running depended directly 
upon the stimulation of the head receptors. Just as painful stimulation of 
the skin releases energy and heightens the ivity level of an animal so, 
we assumed, stimulation of the tongue with sugar releases more energy in 
behavior than stimulation with casein. 

This interpretation is probably incorrect. On every run in the present 
experiment the direct stimulation of the tongue came after the approach 
to food had been completed. If any effects of such stimulation could modify’ 
the time of approach to food, that effect would have to appear 24 hours 
later during the next run. It is highly improbable that the excitatory effect 
of stimulation of the head receptors could be revealed 24 hours later, more 
or less, in the speed of approach to and acceptance of food (p. 282). 


The resolution of this dilemma which Young proposed brings his 
thinking into general proximity with the over-all position of the pres- 
ent volume (see also Young, 1959). However, it will be necessary to 


postpone further discussion thereof to Chapter 8. 


VI. The Effects of Saccharine, Sham Feeding, 
and Stomach Loading 


Here we shall consider a variety of experiments centering on à 
common problem: Is consummatory behavior, such as eating or drink- 
ing, reinforcing from a learning standpoint only if it results in primary 
drive reduction or is such behavior, somehow, reinforcing in its own 
right? [n the preceding section it has been shown that the mere taste 
of food may be reinforcing to a hungry animal, through some presuma- 
bly innate mechanism; but this does not at all exclude the possibility 
that learned processes (involving so-called secondary reinforcement) 
are also prominently involved. Although the evidence is by no means 
yet complete, enough is known to state with some confidence that 
there are purely "habitual" elements in consummatory actions, W hich 
means that these actions may be reinforcing without necessarily in- 
volving primary-drive satiation. : i 

Because a solution of sugar water is highly palatable and nutritious 
to rats and because a saccharine solution tastes much the same but i5 
not at all nutritious, there is here an interesting Opportunity to frac- 
tionate out the influence of hunger reduction from the total cating- 
ingestion constellation. Therefore, in 19:0, Sheffield & Roby cat ricd 
out a series of experiments with saccharine (rats served as subjects) 
which led them to draw the following inferences: 
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UM A non-nourishing but sweet-tasting substance [saccharine] was shown 
in three successive learning situations to be effective reward for instrumental 
learning, its reward value depending on the state of hunger present. 

no The possibility that the sweet taste was an acquired reward rather than 
a primary reward was shown to be extremely unlikely. 
eae The findings demonstrate the expected limitations of Hull's molar 
“need reduction" theory of reinforcement and the necessity of exploring 
indirect reduction of striped-muscle tension as a drive-reduction factor in 
Miller and Dollard's theory of reinforcement. The results are consistent 
with Guthrie's last-response theory of reinforcement, and demonstrate that 
a sweet taste is "reinforcing" in Skinner's system, “satisfying” in Thorndike’s 
system, and "demanded" in Tolman's system. 

4 dt is suggested that elicitation of the consummatory response appears 
to be a more critical primary reinforcing factor in instrumental learning 
than the drive reduction subsequently achieved (p. 481). 


By way of amplifying these statements and of relating them to the 
theories and research findings of others, Sheffield & Roby say: 
that a non-nourishing but sweet-tasting substance 
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A more recent study E 
the finc 


stantiates, these writers believe. 
described earlier by Sheffield & Roby: 

The results strengthen the interpretation that the vigor of the consum- 
matory response is the final determining factor 1n the reinforcing power of 
a reward for instrumental learning. In the present experiments nourishment 
Per se was apparently irrelevant to learning. Reinforcement was in propor- 
tion to calories ingested in the dextrose solutions, but the saccharine con- 
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trols indicate that this is an artifact of the confounding of calories and sweet- 
ness. The position taken here is that sweet stimulation innately clicits in- 
gestion and that this reinforces instrumental learning in proportion to the 
strength of the ingestion response (p. 353)- 


As Sheffield & Roby have indicated, the saccharine technique 
amounts in effect to sham feeding (cf. Miller, 19582, pp- 257-258); but 
they are on less firm ground in insisting that no form of secondary 
reinforcement is involved. They argue that the saccharine itself could 
not have had secondary reinforcing properties (on the dubious grounds 
that the rats had never before tasted anything as swect as the sa ceharine 
which had led to hunger reduction, i.e., real food) and that its rc- 
inforcing effect upon instrumental behavior must, therefore, be in 
some way innately determined. However this may be, the fact is that 
there were many other sources of stimulation accompany ing the 
saccharine consumption w hich bad been followed, often and power- 
fully, by hunger or thirst reduction or both: notably those stimuli 
produced by the act of i ingestion itself. And if these response-correlated 
stimuli had thus acquired secondary-reinforcement power, the con- 
summatory act (and the attendant stimuli) could easily. reinforce be- 
havior antecedent thereto. Stated less rigorously, the idea is that just 

» long as the rats “thought” they were consuming something that 
md allay their hunger, hope would be aroused and would reward 
the behavior that had led up to such consummatory action. 

The older, and more radical way, to carry out sham feeding in a 
laboratory animal is to let it masticate real food but then to have 
the esophagus fastened to a fistula in the neck so that the masticated 
food, instead of passing into the stomach when swallowed, is ejected 
through the fistula. Pavlov (1927) show ed that salivary conditioning 
can be set up in a sham-fed dog just as readily as in a normal one, and 
in an experiment reported in 1951 bv Hull, Livington, Rouse, & 
Barker, it was found that sham feeding can similarly be used to rein- 
force maze or other forms of instrumental learning. But here, as in 
the saccharine experiments, one could not be sure whether it was the 
taste of the food or the consummatory activity as such that carried 
the reinforcing capacity—or, what is more likely, both of them. 
What obviously needs to be done now is an experiment where, in 
addition to being prepared for sham feeding, the subject also has its 
taste receptors deafferenated. If the subject still ate and seemed to be 
rewarded by the reception and mastication of food, it would be clear 
that it is response-correlated stimuli other than taste which are pro- 
viding the secondary: reinforcement for antecedent acts and the mo- 
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mentum or “habit strength” for the eating behavior itself. The fact 
‘tasteless, but still 


that human beings often complain that food is 
cat it, suggests that positive results might be obtained in such an in- 
vestigation. 

‘That the artific 
ing) is rewarding h 


al injection of food into the stomach (stomach load- 
as recently been demonstrated by N. E. Miller and 
co-workers. 

astic tubes were sewn into the stomachs of 
allowed to project from the back of the 
negl. ace od A control solution of normal saline injected directly into the 
stomach had relatively little effect on rate of working for food or amount 
consumed. Food injected directly into the stomach promptly reduced hun- 
ger, and food taken normally by mouth produced an even greater reduction 
(Miller, 1954. p. 151; see also Berkun, Kessen, & Miller, 1952, and Kohn, 1951). 


Here we see further evidence that normal consummatory behavior 
om primary drive reduction; and O'Kelly 
(1954) has obtained similar results in rats motivated by thirst rather 
This investigator reports that rats which have been de- 
1, hours will normally consume an amount 


3 2 . + 
yout 7% of their predrink body weight, but 


is reinforcing quite apart frc 


than hunger. 
prived of water for 2 
of water equivalent to al E ^ i : 
that if this amount of water is injected (under light anesthesia) di- 
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at this effect is not produced by the 


rectly into the stomach 
will not be satisfied and will 
says, “The animals consistently 
predicted on the basis of their 
Control experiments indicate tha À 
anesthesia (sec also Miller. Sampliner, & Woodrow, 1957). N 
There is a common saying to the effect that “eating is just a habit,” 
and there is a well-known (but rather feeble) joke about a man who 
said he believed he could train a horse to go without food, except 
that he seemed to have bad luck when he tried it: the horse always 
died. Food, obviously, is a life necessity; and no matter how it gets 
there, a full stomach is a generally rewarding experience. In fact, 
Coppock & Chambers (1954) have even found that rats are rewarded 
bv intravenous injections of glucose (but not by saline solution). Yet 
: ating there is also an clement of "pure habit" in the 
reinforcement. (cf. Neff, 1953). What this means, 
like other primary drives, is often ac- 
apprehension which can be allaved with- 
accharinc ingestion and sham feeding), 
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the apprehension (as in stomach loading). If, at noon, a healthy hu- 
man being could be made to believe (perhaps by hypnosis) that he 
had eaten a hearty lunch, it is quite possible that he would not be 
unusually “hungry” during the ensuing afternoon; and if, contrariwise, 
a human being (under anesthesia) could be given a good meal with- 
out knowing it just before his lunch time, he would very probably 
eat lunch again. If asked if he were hungry, he might well reply that 
he was not, at the moment, but was “afraid” he might be in the after- 
noon if he did not cat as usual. 5 

Further substantiation of this point of view has been advanced by 
Ashby (1952): É 

Thus it has been shown that dogs with an esophageal fistula, deprived of 
water for some hours, would, when offered water, drink approximately the 
quantity that would correct the deprivation, and would then stop drinking; 
they would stop although no water had entered stomach or system. The 
properties of these mechanisms have not yet been fully elucidated; so train- 
ing by reward uses mechanisms of unknown properties (p. 113). 

It is believed that we are now approaching an improved under- 
standing of these mechanisms. 


VII. The Problem of Motivation, Activity, 
and Appetite 


At this point in our analysis we come to what is probably the most 
important and difficult issue in this entire volume. If it can be resolved 
satisfactorily, the rest of the chapters follow almost as a matter of 


course; if not, the ensuing discussion is fraught with ambiguity and 
uncertainty. : 

Already a number of writers have been cited who have hinted at this 
difficulty, and it has probably become even more explicit in the read- 
er's mind. The dilemma, briefly put, is this: The present analysis as- 
sumes that type-2 secondary reinforcement (hope) consists of a re- 
duction in drive fear, cued off by some stimulus or situation which 
has commonly been associated with primary-drive reduction, whereas 
common observation suggests that the appearance of such a stimulus 
Or situation often produces in the subject, not a relaxation (as the 
notion of drive-fear reduction would seem to require), but a state of 
excitement and heightened activity? 

The problem may, in part, arise from a misconception: because 


"For a presentation of the general background of this discussion, see Brown's 
(1955) paper, “Pleasure-secking behavior and the drive-reduction hypothesis.” 
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(1950), it was found that rats performed at a eles devel d in an ae 
tivity wheel when hungry than when not hungry; ; and this, among 
similar observations, has led to the widespread inference that activity 
and drive are, of necessity, covariant. But Sheffield & Campbell (1954) 
have made a more detailed analysis of the situation, along the following 
lines: 


In a previous study (Campbell & Sheffield, 1953) it was shown that in a 
highly constant environment food deprivation for even 72 hr. produces 
only a very slight i increase in activity compared to the increase during 10- 
min. periods i in which the visual and auditory environment is altered. More- 
over, the magnitude of the increase in activity produced bv the environ- 
mental change was related to hours of deprivation in a müch more pro- 
nounced way than was the so-called "spontaneous" activity. These results 
Ae e interpreted by the authors as evidence that hunger. does not func- 
tion as a stimulus to activity but instead is a state in w thich thresholds of 
normal activity responses to internal and particularly to external. stimuli 
are lowered. The purpose of the present study was to investigate | these 
suppositions] under controlled conditions (p. 97). 


In summarizing their study, Shefficld & Campbell sav: 


Using stabilimeter-type activity cages, the increase in activity in hungry 
rats produced by sudden change in an otherwise constant environment 
was recorded once each day for a s-min. period of the environmental 
change. For one experimental group and one control group the environ- 
mental change consisted of shutting off a masking sound and turning on 
lights in normally dark environment; for a second experimental group and 
control group the change consisted of turning off the sound and turning 
off the lights in a normal lighted environment. For the experimental groups 
the 5-min. of change in the auditory and visual fields were always followed 
immediately by automatic dropping of the daily food ration into the ac- 
tivity cages. For the control groups the change was unaccompanied by food 
—their daily ration was dropped at a different timc. 

Experimental animals showed a progressive rise over a 12-day period in 
the amount of activity during the 5-min. stimulus change leading to food 
{Fi ig. 6-2], while control animals showed a decline over the 12 days. Also, 
when the activity during the 5-min. interval was plotted minute by minute 
in the experimental groups, it was found that the acquisition of increased 
activity during the last 3 days of training was greater in later portions of 
the interv al—shortly before food arrived (p. 100). 


Nothing, it seems, could be empirically clearer or logically neater 
than these results. They show, quite unambiguously, that hunger, 
per se, is not a drive to activity but that a stimulus or situation change 
which forecasts the arrival of food releases a very marked increase 
in activity. Here, manifestly, a discrimination is being established (al- 
though Sheffield & Campbell interpret the situation otherwise): the 
rats learn that most of the time there is no point in moving about (in 
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search of food), and during this period they are, relatively speaking, 
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However, there are purely subjective grounds for supposing that 
the problem is not so easily disposed of as the foregoing discussion 
might suggest. Appetitive excitement, or "passion," in the region of 
a goal is a universal experience and is particular striking in the realm 
of sex (where the primary drive is comparatively weak and the 
“arousal” element prominent). However, in order to keep the discus- 
sion as simple and clear as possible, let us take as a paradigm the ap- 
petitive or excitement reaction as it occurs in connection with the act 
of urination, An individual may be for some time vaguely aware of 


bladder provura (primary drive); and when, at length, hc finally de- 
cides to "relieve" himself, he may experience a sudden and quite 
powerful accretion in the “need.” As he approaches the act itself, the 
over-all motivation may become highly impelling and the thought 
of not proceeding almost intolerable. Here, it seems, is a bona fide 
instance where stimuli (and actions) previously associated with 
primary-drive reduction acquire the capacity to clicit an unmistakable 
rise in total motivation, under circumstances which ought, at the 
same time, to invest them with the power of secondary reinforcement. 
This, then, is the type of evidence which makes some writers hesitant 
to accept the view that secondary reinforcement involves drive 
decrement. Spence has already been quoted (Chapter 5) as favoring 
an excitatory (rather than relaxation) theory of secondary reinforce- 
ment; and Seward (1956b) has recently put the predicament thus: 


Drive reduction seems to be a sufficient condition for response selection. 
But rewards apparently do more than reduce drive; as incentives thev ap- 
maha uaa : : A 

pear to heighten excitement as well as activate instrumental responses. This 


“Although these experiments show that drives do not inevitably goad the animal 
into activity, they do not rule out the possibility that drives are strong stimuli. 


. It will be remembered that my definition of drives says nothing about activity, 
hur rather deals with increasing the performance of responses rewary led by the 
offset of the drive or by the goal objects that produce satiation. . . . Suppose that 


at unpredictable intervals, without providing any stimulation, the experimenters 
had introduced food alternatively at opposite sides of their activity apparatus 
throughout the test session. Then the animals would have been reinforced. for 
shuttling on a variable-interval schedule. We would expect this shuttling, which 
would be recorded as activity, to increase with increasing hunger just as docs 
the rate of bar-pressing. . . . In short, we can only predict thar the subjects will 
be active to a pattern of stimulation including strong drives whenever activity 
has been reinforced to that pattern. The situation used by Sheffield & Campbell 
involve long periods during which activity was not reinforced to the pattern 
of drive stimulus plus apparatus without the sound of the food-delivery. mecha- 
nism” (pp. 255-256). The only question in this discussion would seem to hinge 
upon the definition of “stimulus.” This problem will be reverted to at various 
points later in this book. 
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poses a problem: How can drive reduction and incentive induction both 
produce the same result, i.c. strengthen à concurrent response? (p. 202). 

In an etfort to resolve this paradox (one wonders how long it took 
Nature, in the course of organic evolution, to resolve it!), Seward 
) primary reinforcement involves a re- 
duction in primary motivation, that (b) secondary reinforcement 
type-r involves a drop in secondary motivation, but that (c) secondary 
reinforcement. tv pe-2 involves an increase in what he calls tertiary 
motivation. While this formulation seems to fit the raw facts rather 
exactly, there is the difficulty (w hich Seward himself well recognizes) 
that it takes one away from the homeostatic mode of thought (which 
has otherwise proved so useful) toward a type of theory that is con- 
ceptually far from neat and satisfying. A sort of “last-ditch” attempt 
Will be made in the next section to suggest how the phenomenon 
accommodated within the general homeo- 


(1950) has suggested that (a 


of appetite arousal can be 
static framework. 


VIII. A Possible Resolution of the 
Basic Dilemma 
made some preliminary sug- 


ago the present writer 
i=] A ; 
paradox which will be 


"tertiary-motivation" 
more detailed analysis that is to follow 


A few years 
gestions concerning the 
used as the basis for the 
shortly: 
is one of the essential characteristics 
v evident that many of their ac- 
X negative feedback, as just defined | see Mowrer, 
tivities or functions serve to hold certain states 
iation. Positive feedback, when 
unds as if it would be patho- 


asis, or self-regulation, 
ms, it will be immec 


Since homeos' 
of living organis 
tivities have the properties ¢ 
1960, Chapter 7|. These ac 5 Ol 
or qualities within fairly narrow limits of vi 
the concept is applied to living organisms, So i 
logical, to say the least, and, in the extreme case, lethal. Actually, we find 
Some very instructive instances of it in living organisms. —— ] . 

We have already seen that there isa general tendency for incidental stimuli 
which have been associated with primary-drive reduction to produce sec- 
ondary-drive reduction and for such stimuli as have been associated with 
primary-drive induction to produce secondary-drive induction. These two 
types of contiguity learning, or conditioning. unquestionably tend to pro- 
duce (indirectly, through the integrating function we call consciousness) 
responses which have biological utility —notably, flight or immobility in 
the face of danger and approach to objec s or situations W ith rewarding 
potentialities. However, there is one crucial respect in w hich this general 
Scheme is inadequate. non-biological. The world’s good things do not always 
remain conveniently at Testy waiting to be claimed and consumed. Some- 
times these goal objects, especially when they are other organisms, have 
a way, at the critical moment, of eluding their pursuers. Therefore, an 


cl 
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organism hae became more and more confident and rina as it it approached 
a quarry might find itself slowing up at just the point when a final * push? 
was needed for success. It is therefore interesting and altogether. under- 
standable that we should find the phenomenon of appetite. Its outstandit 
feature is an increase in secondary motivation just as consummation is im- 
minent, thus giv ing to behavior at this crucial moment a peculiar urgency 
and ' ‘oomph.” 

Here, it seems, is an instance of positive feedback: “The nearer vou get, 
the more you want it" carries the proper connotation here. C certainly such 
an arrangement is biologically intelligible, but it is nonetheless enigmatic. 
Neal Miller (1951b) has put the matter this way. We have reason for 
thinking that stimuli associated with consummatory states take on cont 
dictory capacities: a tendency to cause a decrement in secondary drive 
(secondary reinforcement) and a tendency to cause an increment in sec- 
ondary drive (which we call, not punishment, but appetite). As Miller 
points out, if these two tendencies occurred simultaneously, they would 
be self-cancelling, mutually neutralizing, so his proposal is that they may 
alternate, producing intermittent. bubbles of pleasurable anticipation (re- 
laxation) and surges of intensified (appetitive) drive. This is frankly a 
speculation and, even if true, leaves many unanswered questions. However, 
consideration of the problem in the eyberncties setting illuminates and usc- 
fully sharpens it (Mowrer, 1954, pp. 86-87). 


Granted that the evidence on this complicated issue is not vet all 
in, at least a provisional resolution seems possible. A stimulus which 
has secondary reinforcing properties is one which, per hypothesis, 
is capable of reducing some sort of secondary drive, most commonly 
drive-produced fear. “Now fear reduction presupposes a lessening of 
activity on the part of the sympathetic division of the autonomic 
nervous system. But we know that stimuli associated with primary 
drive reduction may be not only reassuring and relaxing but also ex- 
citing, stimulating, motivating. How are these seemingly contradic- 
tory effects possible? May it not be that stimuli w hich in the past 
have been associated with consummitory states acquire the capacity 
first to allay fear, i.c., quiet sympathetic arousal, and then to activate 
the parasy mpathetic nervous system, which mediates the appetites? 

It has often been remarked that the sympathetic and parasy mpathetic 
divisions of the autonomic nervous system are antagonistic or reciprocal 
in their actions; and it is certainly true that fear must subside before 
one can react appetitively and that one has to stop being appetitively 
aroused before one can become very fearful.'2 However, there is a 
good deal more that can be said about this relationship, if it is 


ò 


"Brown (1955) cites Gantt (1949) for the observation “that anxiety symptoms 
in neurotic = are completely inhibited by sexual stimulation even prior to or- 
gasm” (p. 175). See also Section VII, Chapter 5. 
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schematized as indicated in Fig. 6-3. Here it is suggested that there is 
pe only a "vertical" ( quantitative) but also a "horizontal" (qualitative) 
à iens of function. This means that a rise or fall in either sym- 
pathetic or parasympathetic activity involve a rise or fall in motiva- 
tion, Bur—and this is the point especially to be noted— "movement" 
along the functional course indicated in Fig. 6-3 is never directly up 
or down, there is always a lateral component, such that the individual, 
in becoming either more or less fearful or more or less appetitively 
aroused, is also moving either from left to right (away from privation 
i from right to left (away from consumma- 
~ although a decline in fear and a 
f view, antithetical, in another 
they are two successive 
ation to consum- 


tow ard consummation) or 
un ON, ard privation). Therefore, 
rise in appetite are, from one point o 
a continuous function: 
gress from priv 
in reverse order, of the 


Way they represent 
Steps or stages in the organism's pr 
mation. And, of course, ‘the same holds truc. 
relationship between a decline in appetite and a rise in fear. 

Classical hedonic theory (see Section V) tended to assume that 
stimuli are inherently pleasant or unpleasant, quite apart from any 
\ s, in homeostatic theory. the tendency is 
drive reduction and the opposite quality to 
ese TWO systems, in appearing to be 
s cach referring more partic- 
o different, but related, 


factor of intensity; where 
to equate pleasure to 
drive increase. It may be that th 
Contradictory, are really complem 
ularly to one rather than to the oth 


entar 
er of the t% 


dimensions implied in Fig. 6-3- 
3E this analysis is sound, that a secondary re- 
ges upon an organism in a state of strong 
c auses a relicf (fear reduction) reac- 
(excitement) reaction, as 
ass through, for reasons 


; Hence we conclude, 
Inforcer, when it first impin 
drive and drive fear (privation). c 
ton and then, very probably. an appetitive 
a sort of detour which the organism has to P: 


already indicated. pm is Way [9 2 state of consummation and satia- 
i is motivating and, 


ton, This is not to deny that appetitive arousal is i 
IN a sense, even punishing. If, for example, an individual in a state of 


Complete satiation and comfort could, merely by wishing Ms by 
pressing a button). suddenly tr aself back to a condition of 
appetitive arousal (without the normally attendant. primary drive 
also being present), it is unlikely that he would do so. Why, then, when 
4 or otherwise deprived organism is at point X on 
does it not re 
consummation? This may, in 
als who are said to be shy. 


individual is willing to 


ansport hin 


a hungry, thirsty, ; 
the CON Cc sho nF main there, instead of mov- 
pE on through appct 
act, be exactly what h 
But the more confident. 


ig. 6-3. 
titive arousal to 
appens in individu 
more experienced 
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tolerate appetitive arousal because it promises to lead, ultimately, to 
consummation; whereas the rise in motivation which is produced by 
fear points in the reverse direction, toward further privation ( Chapter 
11). Or, to put the matter a little differently, movement in cither 


| i 
s Privation 
HF 

zm 
E Sympathetic Parasympathetic 
t arousal „~ arousal 
E: 
c 
S 
E 
= 
9 
a P 

= Consummation 

| | ] 1 ! 1 1 1 ! ! | 


Motivational quality 


Fig. 6-3. Diagram suggesting a hypothesis concerning the relationship between 
secondary reinforcement, conceived as fear reduction, and appetite. It is true 
that fear reduction (lessening of sympathetic activity) represents a drop in mo- 
tivation and that appetite induction (parasympathetic arousal) represents a rise 
in motivation. Therefore, in onc important sense, the two processes are antithetical. 
However, both fear reduction and appetite induction represent lateral movement 
in the same direction, namely to the right. Further implications of this scheme 
are discussed in the text. Y 


direction from point X on the curve involves a form of “loss,” i.c. 
drive increase; but appetitive arousal also involves a form of “gain,” 
Le, movement toward consummation, whereas fear arousal involves 
a "loss" in both dimensions." 

What then, in simplest terms, may one say of the concept of sec- 


"In like manner one may ask: Should not disappointment be experienced as 
rewarding, since it presumably involves a drop in motivation. Again, for shy 
individuals it may be perceived in somewhat this way: “I am really glad I didn't 
get it.” But, in the more usual situation, disappointment is not perceived in this 


way for this reason: Just as appetitive arousal, while mildly “punishing” in its 
own right, is at the same time premonitory of good things to come; disappoint- 
ment, while mildly "rewarding" in its own right, is premonitory of bad things 
(want, privation, erc.) to come, thus more than cancelling out such indigenous 
reward value as it may have. 
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or of hope? One may say: (a) that 
is moving to the 
is in a state of 


ondary reinforcement: type-?, 
whenever an otherwise strongly motivated organism is 
right, along the curve show in Fig. 6-3, that organism 
(b) that such movement may involve either 
asc) or appetitive arousal (second phase), 
ally) of a state of consummation, 
: anything achieved beforehand, 


hope or hopefulness; and 
drive fear decrement (first ph: 
followed by the attainment (norm 
with a general drive level well below 


at point X.* 
It is to be regretted that this hypothesis is somewhat intricate, but 


As of the present, this hypothesis fits 
accepted as the basis for further 
ndicated. 


S0, apparently, are the facts. 
the facts rather well and will be 
theoretical analysis until revision is clearly i 
IX. Homeostasis, Hedonism, and 
Brain Stimulation 

-annot end without reference 
as recently challenged homeo- 
Electrical stimulation of 


Although already long, this chapter 
to an experimental phenomenon which h 
static theory in a particularly dramatic wav. | 
the central nervous system for experimental purposes has, of course, 
‘ And the classical question which underlay 
If stimulation is applied ata particular point, 


a relatively long history. 
such work has been th 


helpful to suggest that, in the movement of an organism 
continuum shown in Fig. 6-3. there is what may be 
that occurs at X. Fear is the normal accompaniment 
t of consummation is in sight (equals, no hope). 
and the organism sees itself moving toward 
shes a low point at X. However, in 
ation which could be so disastrous at 
second or auxiliary motivational mecha- 


“Te will perhaps be 
from left to right along the 
called a qualitative change 
of strong drive when no prospec 


But then if circumstances. change 
ar diminishes and ri 


consummation, its fe 
order to avoid the let-down n total mo 
this crucial point (sec above), it is as if 


nism of a different (appetitive) kind goes ! J ; c 
in the nature of affective arousal at X. In trying to 


ithor once took the example of a hungry dog 
© but “worried.” Then he spied a 


nto operation. Hence, the reference 


to 4 


"qualitative" change 
make this point with a class, the at 
Which, in the beginning. Was not only hungry 
rabbit, started chasing it, gradually gained on it, 3 
began to anticipate how rabbit was going to taste. Here, presumably, is 
inge in emotional arousal alluded to. But then a student asked: 
į rabbit during this same time?” This side of the 


. and, as his hunger fear dissipated, 


good the 


the qualitative cha 


“What was happening te the t ner 
1 one. Since the rabbir was "moving," not toward con- 


of course, à sac r no 
his fear was presumably rising and would terminate 


story is, 


summation but consumption. s 
in terror, Any reader who likes rabbits better than dogs may therefore supply 


a different ending: just in the nick of time the rabbit got to a safe hole in the 
ground and the dog didn't have him for dinner after all (relief for the rabbit, dis- 


appointment for the dog)! 
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what response (or sensation) will it produce? However, in 1954, Olds 
& Milner reported an investigation in which they had, in effect, asked 
this question in reverse: If an organism has already made a response 
of some sort, how will brain stimulation then affect. the organism's 
subsequent tendency to repeat or inhibit this response? Because clec- 
trical stimulation. (shock) applied to the cutaneous receptors of thc 
body (and to the underlying muscles) has a tendency. to inhibit re- 
sponses which it follows, it was natural to assume that clectrical 
stimulation, centrally applied, would have the same effect. But such 
was not found to be necessarily the case. If rats with electrodes 
chronically implanted in the septal arca of the brain were given an 
Opportunity to press a small bar and thereby "shock" themselves, 
centrally, they would do so with astonishing regularity and persistence. 
Here, obviously, was a first-rate enigma—and_ one whose resolution 
may prove highly instructive. 

One of the most immediate effects of the Olds-Milner. discove KB 
has been to prompt a careful reconsideration of the question: What, 
precisely, is "pleasure," "reward," “reinforcement?” As previously 
indicated in this chapter, there have been, in general, two types of 
answers, with hedonism holding that this phenomenon occurs when- 
ever certain distinctive sensations are aroused, and the homeostatic 
type of theory holding that it occurs only when there is a major 
diminution in stimulation or drive (“adjustment”) of some sort. At 
first blush, hedonism fares better than does homeostasis when rc- 
examined in light of the Olds-Milner effect: If reward is simply a 
matter of certain types of sensation, this phenomenon should, rcasona- 
bly enough, be reproducible bv artificial stimulation of cither the 
appropriate peripheral (afferent) nerves or by appropriate central 
stimulation; wherea 


since reward, homeostatically considered, in- 
volves a reduction in stimulation, it is less readily conceivable how it 
could be recreated by stimulation.” However, as we shall sce, there 
are some other interesting and plausible possibilities in this connec- 
tion which, in view of the general difficulties in hedonistic theory 


already delineated in this chapter, are well worth considering. 

One of these possibilities has recently been explored by N. n Miller 
( 19s9b). He points out that if any drive of natural origin is present 
when septal brain stimulation occurs, the latter might well produce 
its demonstrated reinforcing effect by at least bricfly obliterating the 
central representation of the drive—or, as Miller say s. by "temporarily 


© Olds & Milners own comment in this connection is: “The ‘Clinical impression? 
of the [experimenters is| that the phenomenon represents strong pursuit of a posi- 


tive stimulus rather than escape from some negative condition” (1ys4. p. 426)- 
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; al sources of stimulation. Such a functional 
climination of motivational stimuli might occur during unconscious- 
ness produced by a petit mal seizure" (p. 25 cÉ Brady, 19582). 

And this surmise has received quite specific support from a paper 
recently. published by Nielson, Doty, & Rutledge (1958). Addressing 
themselves to the “reasons why central stimulation should be reward- 
ing" (p. 427), these writers conclude: 


e 


blocking out motivation 


; induce sustained self- 
apparently rewarding 
c-like neural action. 
if such it is, should 


riments necessary tc 


All procedures in these. expe 
leus suggest that the 


stimulation of the caudate nuc 
effect for the animal could be based on excessive, seizur 
-. . One cannot say why such excessive neural action, 
be rewarding; but this element is present in the phenomenon of orgasm, and 
the self-induction in children of petit mal attacks by photic means (Robert- 
Son, 1954) is not unlike the behavior of self-stimulating cats (p- 43»). 


The: reason why Nielson Days > Rutledge reg rd the electrical 
C Olds cffect as "excessive" is that they have 
of the stimulation here required is con- 
ary to produce a central “sensation,” 


sti ati ; ? 
e PEN involved in the 
demonstrated that the intensity 
sider; ; A 
iderably greater than 15 necess 


as detected by the conditioned-r uc. They say: 


esponse techniq 

"The intensity. | required. to produce the Olds effect] must be far above 
that required for perception as CS and the stimulation must be prolonged 
or available in rapidly succeeding burs High intensity stimulation appears 
to be a common factor in other experiments of this type (Olds & Milner, 
Boren, Conrad, & Sidman, 1957) and rapidly 


1954; Olds, 1956c rady 
5 s, 1956c; and Brady, x 
o: ' i of structures of the limbic system are 
f the neural struc- 


ation 


repes e ^ . 
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sought in monkeys (Lilly, 1958). The W ide distribution o iral s 
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€ d ise Ja 
ME non-specific, convulsion- 
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t, Miller (amor p 
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the intensity of certain drives. The fact that the effect of certain drives is 
at least partially specific to the reward effect of stimulation in certain loca- 


i Tu periments (p. 2)- 
Uons opens up the possibility for further experimer (p. 2) 
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becomes translated from mere stimulation termination over into a 
more “positive, " more "active" state of some kind and that it is the 
latter which, in the final analysis, is the agent that fixates response 
tendencies. This being true, the Olds-Milner effect becomes not in the 
least surprising: all that is neces ary ds that a part of the brain be 
found in which the final, active reinforcing state can be aroused, not 
by cessation of natural stimulation, but by an artificial electrical 
stimulus. Since reward, in the homeostatic ‘sense, can of course be 
experienced as a result of the satiation of many different forms of 
drive and discomfort, it follows that there may w “ell be many different 
sites in the brain where this sort of transformation takes place. Say 


Olds & Milner: 


There are numerous places in the lower centers of the brain where clec- 
trical stimulation is rewarding in the sense that the experimental animal 
will stimulate itself in these places frequently and regularly for long periods 
of time if permitted to do so. It is possible to obtain these results from as 
far back as the tegmentum, and as far forward as the septal area; from as 
far down as the subthalamus, and as far up as the cingulate gyrus of the 
cortex (1954, p- 426). 3 


When J. Konorski, of the Nencki Institute of F “xperimental Biology 
(Warsaw), was in this country early in 1958, he described unpublished 
experiments recently conducted in Sweden in which it has been found 
possible to stimulate the "thirst center" of goats electrically ( Andersson 
& Wvyrwicka, 1957). A normally thirsty goat is first trained to push 
a treadle with one of its front feet as a means of obtaining a small 
amount of water. After the habit is well established, the animal is al- 
lowed to drink to satiation and then the thirst center is stimulated. 
The immediate result is that the animal starts pushing the treadle as a 
means of obtaining water (sec also Miller, 1957). This shows that the 
subject is being made genuinely thirsty, rather than that some "drink- 
ing reflex" is merely being stimulated. If the experience of thirst can 
be thus artificially aroused, it is by no means improbable that the ex- 
perience of thirst (or other drive) reduction can be likewise produced. 

In the literature there has already been some speculation about the 
possible existence of a “joy center,” which, quite unjustifiably, has 
been interpreted as being more congruent with the presuppositions 
of hedonism than with those of homeostasis. If the notion advanced 


in the preceding paragraph should be valid, such a center or centers 
would follow as a matter of course. Relief or reward, as centrally 
registered, need not at all be the mere absence or cessation of excitation 
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but a decidedly active state of affairs which might be tripped off by 
just the procedure which Olds & Milner have hit upon. 
Ziegler (1957) summarizes prevailing opinions in this connection 


as follows: 


discovery of positive reinforcement produced 
by intracranial self-stimulation has aroused considerable interest and a great 
deal of speculation. This speculation ranges from Delgado's concept of "at- 
tractive” and “unattractive” cerebral areas (Delgado & Burstein, 1956), to 
Olds’ tentative conclusion that “stimulation in these areas might excite some 
of the nerve cells that would be excited by satis action of the basic drives 
—hunger, sex, thirst, and so forth” (Olds, 1956b). The analysis of neural 
and behavioral mechanisms meditating the reinforcing effects of intracranial 
self-stimulation has hardly begun. Indeed, it has not yet been conclusively 
demonstrated that we are dealing with “pleasure centers” or “a system within 
the brain whose function it is to produce a rewarding effect upon behavior” 


(Olds & Milner, 1954) (P: 373) 
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Moreover, an experiment recently reported by Sidman, Brady, 
Boren, & Conrad (1955) calls attention to the possibility that the Olds- 
Milner effect is highly anomalous and not at all the same as that in- 
volved in normal reward learning. At least they find that a Skinner 
bar-pressing habit which is established under the influence of reward 
provided by food or water is typically inhibited by a tone which has 
previously been associated with electric shock administered to the 
surface of the body, whereas a bar-passing habit established by the 
Olds-Milner method is not so inhibited. And Miller has shown that 
although rats and other laboratory animals will develop habits if briefly 
stimulated by the Olds-Milner method, they will also learn to make 
a response which taurus off this form of stimulation if it is appreciably 
extended.!* 

In light of these varied and not entirely consistent experimental 
findings, suffice it to say, for the present, that the exact import of 
the Olds-Milner effect is not yet known; certainly there is nothing 
in the now-known facts which argues compellingly against the general 
biological and psychological principle of homeostasis (Stagner, 1951; 
Dempsy, 1951; R. C. Davis, 1958). One might well wish that the whole 
s; but since homeostasis 


matter were less up in the air than it presently 
accounts for such a very wide range of behavioral data and since the 
apparent exceptions involve many ambiguities, it seems that the system- 
ng, has no choice but to proceed on 


atic theory builder, as of this w 
the assumption that this model is basically sound and will, at most, be 
refined and corrected in minor ways rather than totally invalidated, 

On the whole, it can be said that recent studies of the reticular 
mechanisms lend strong support for the systematic position which 
this book is attempting to portray. For example, $ Samuels (1939) con- 
cludes a long review on “Reticular Mechanisms and Behavior" as fol- 


lows: 


Studies of the reticular formation indicate that its structural complexity 
and functional plasticity. override the conceptual limitations inherent in 
more static, reflex-like neural mechanisms. These characteristics permit it 
to exert facilitatory and suppressive effects which have a time-course of 
seconds and even minutes on the activity of central nervous system struc- 
tures. This span is comparable to that of many behavioral events. 


" Penfield (1954), Heath (1953). and others have extensively stimulated. various 
anesthetized human beings and have 


areas and surfaces of the brains of locally 
obtained elaborate. introspective reports (see also. Mowrer, 1960. Chapter 7). It 
would seem that the information thus derived could be used to clarify the issues 
discussed above. To date, apparently, no systematic effort has been made to col- 
lare knowledge obtained by the two different approaches. 
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an attempt to ia od psychological constructs be critically evaluated in 
Specific" as w be pode ies extent to which they are a function of “non- 
dicate that c AS specific factors. Conceptual reassessment May well in- 
clusive Neg heu. E now regarded as independent and mutually ex- 
basis of a ane ot operational criteria are functionally interrelated on the 

von factor of reticular activation (p 20, italics added). 

t with our 2 ssumption that every 
activates à relatively 


Thiet 
€ foregoi . à : 
regoing is eminently consisten 


& 
specific" IT 
ic T 5 ; , 3 
drive state of considerable intensity 


" 
nons as 
; HORNE? AER s : à ; 
pecific" affective state (of renston. arousal, apprehension) which 
ioned. (both as regards onset and offset, 


IS à ; 
Sen qa uf being condit 
tion ue a relaxation) to formerly 
rM by (as shown in Ch 
related to à "i Wastüditg gricf and anger). 
remarkably " ‘ response-dependent and independen l 
N e exible and comprehensive explanatory system. 
theory s point, Conant's ( 1947) famous dictum comes to mind, that a 
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READS 
the per T ted. It is believed that the: 
plexities delineated in this chapter: 


Revised Two-Factor Theory 
and the Concept of Habit 


In some ways, the present chapter is the most important one in 
this book. In it a hypothesis will be made explicit which is central to 
the book as a whole. The six preceding chapters have been stepping 
stones for the development of this hypothesis; and the chapters w hich 
follow are, in the main, merely amplifications or applications thereof. 
But for the thoughtful reader, this chapter is almost certain to be 
anticlimactic: he will already have anticipated at least the essence of the 
point of view which is now to be presented, for it is broadly implicit 
in what has gone before. However, before entering upon the discussion 
of the conception of “habit” which will constitute most of this chapter. 
it will be useful to review, in highly synoptic form, the argument as it 
has developed up to this point. a 


I. Recapitulation and an Inference 


Although the foregoing chapters have topical, or logical. headings, 
the reader will have noted that they are also organized historically, 
chronologically. In order to make both of these principles of organiz 
tion more explicit and to indicate more clearly how and why this 
type of analysis generates a special way of viewing the phenomenon 
of habit, the high lights of the preceding discussion are presented in 


Table 7-1. 
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TABLE 7-1 


Historical and Logical Antecedents of Present Version of Two-Factor 
Learning Theory 


Learning Theory, Original Version 

Habit formation (Thorndike) 
Reward and punishment 
Response substitution 


Tazo-Factor 
Conditioning (Pavlov, Bekhterev) 
Positive and negative unconditioned stimuli 
Stimulus substitution 

Hull's Drive-Reduction Theory 

A monistic theory of reinforcement purporting to 

explain both conditioning and habit formation. Did 

not deal adequately with avoidance behavior (ac- 

tive and passive) or with secondary reinforcement. 


Version Two 


Tro-Factor Learning Theory, 
Solution learning (habit formation) 


Sign learning (fear conditioning ) 
Central nervous system and skeletal 


Autonomic nervous system and vis- 
ceral and vascular tissue 
avoidance behavior, to be adequately 
both sign learning and so- 
aot, however deal ade- 


musculature 


Showed that 
explained, must involve 
Theory did r 


lution learning. 
larv reinforcement (tv pe-2) 


quately with seconc 


or with the concept of habit. 


sent. Version of T* o-Factor Theory 


Decremental reinforcement (reward) 


Pr 
Incremental reinforeeme 
ment) 
Primary reinforcement 
Secondary reinforcement 


nt (punish- 


Primary reinforcement 
Secondary reinforcement 
Danger signal off Safety signal on 


Safety signal off 
(relict ) (hope) 


(dis appoint- 
ment) 

Here it will be noted that all learning is (by implication) conditioning, so 
that the theory remains “two-factored” only with respect to the forms of 
reinforcement. involved, 1€. incremental and decremental. Since cach of 
these may involve primary drive or cither of two forms of secondary drive, 


Danger signal on 
(fear) 


there are thus, in effect, LV operationally distinguishable forms of reinforce- 
be associa 


ment; and cach of these may 
response-dependent stimulation. Therefore, if this classification is so defined 
as to include the nature of the associated stimuli, there are twelve "kinds" of 


The reader is encour 


ed with either independent or 


FE OSAAN aged to test this classification for ex- 


haustiveness. 
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As previous chapters have indicated, there is now widespread 
agreement concerning the nature of avoidance learning, both active and 
passive. As suggested in Table 7-1, there are three ways in which 
acti A 
can occur. If an independent, environmentally produced. stimulus is 
followed bv (a) primary-drive increment, by (b) secondary-drive 
increment (fear ty pe). or by (6) secondary drive increment (disap- 
see Chapter s, Section VII), then that stimulus will 
y to drive and keep the subject away from it. That 


avoidance, and three ways in which passive avoidance, learning 


pointment type; 
acquire the capac 
is to sav, active avoidance learning can occur on the basis of ordinary 
punishment ( (infliction of pain). threat. (warning), or disappointment. 
And if response-correlated stimulation is followed by any of these 
three forms of incremental reinforcement, passive avoidance learning 
(i.e. response inhibition) will ensue. Here theory seems to be adequate 
to the facts, and the facts logically exhaust the theory. 

In like manner, if an independent, environmentally produced 
stimulus is followed (a) by primary-drive decrement, (b) by second- 
ary-drive decrement (relief type), or (c) by secondary-drive decre- 
ment (hope type), then that stimulus will acquire the capacity to 
attract and bold the subject to or near it, thus accounting for the 
known forms of active approach behavior. But what happens if any 
of these three forms of decremental reinforcement is associated with 
response-correlated stimulation? Here is an area of the total system 
which is manifestly undeveloped and unesploited. Can our surmise be 
correct that therein lies the possibility for an advance in the under- 
standing of habit as radical and as far-reaching in its implications as has 
been the change from “Thorndike’s "stamping-out" conception of 


punishment. to the present view thereof? 


The Phenomenon of "Habit" Reinterpreted 


Perhaps the best way to move forward with this discussion is to 
recall what has been said in Chapters 4 to 6 about what happens when 
a stimulus. or signal, which is external to the organism, immediately 
precedes primary-drive reduction, We know that such a stimulus itself 
soon becomes capable of serving as a so-called “secondary” rcinforc- 
ing agent. It can be used to produce higher-order conditioning, of the 
type ‘studied by Pavlov in his salivary experiments; and it can be used 
to fixate a particular bit of so-called instrumental behavior, as in the 
Saltzman and Zimmerman experiments (Chapter 4). And more than 
this, we know (or at least suspect) that a stimulus with secondary- 
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reinforcement potency is effective as a reinforcer by virtue of its own 
pacity to produce drive reduction, i.c., reduction in some species 
of the secondary drive of fear (Chapters 5 and 6). 

ackground of this knowledge, let us ask: What 


Now against the b 
would happen if a stimulus, or stimulus pattern, which is to be followed 


bv primary-drive reduction, instead of originating in the external 
! response-produced? In other words: What would 
cinforcement properties got attached to the 
tactile, proprioceptive, and other stimuli characteristically associated 
with the occurrence of a given response? A commonplace example 
provides an instructive answer. All boys who ever lived in the country 
(and even some city boys) know how to teach a dog to “shake hands.” 
You do not wait until the dog “just happens" to make the response in 
question and then rew ard its occurrence; that would be too slow and 
uncertain a procedure. Instead you take the dog's right forepaw in 
vour right hand and flex the leg acutely. At the same time, vou give 
the doga morsel of some preferred food. If the dog is at all bright and 
friendly, vou soon have the isfaction of sccing him spontaneously 
"offer" to "shake hands" with vou. What has happened? You have 
taught the dog a “habit.” And he has learned it, not by "doing" any- 
thing, but by having something done to him. It is not that the dog's 
hunger initially prompted him, in t 
- is foot in such a way as to prompt you 


"random," behavior, to lift hi à 

fo reward him You lifted it for bim. In other words, vou did not 
reinforce an S—R connection, or "bond," between hunger (Sa) and 
paw lifting (R;). What happened instead, it seems, is that the training 
procedure established a "connection" between certain stimuli, aroused 
by passively moving the dog's leg. and secondary reinforcement, type- 


tation that food would be alone shortly. And 


environment, were 
happen if secondary-r 


in the course of trial-and-error, or 


2, ie., the hope or expec 


What is the result? Soon the dog is 5 L 
“begging” for (showing he wants) food. The 


a bond, connection, or "reflex" connecting 


"doing it himself," making the 


response as à means of 
dog thus acts as if there were d i ; 
hunger with paw lifting; but the circumstances under which the learn- 
ing has occurred give us pause- 

That the example just cited i ona dica 
by the fact that human language behavior, at least in its beginnings 
in the infant, is similarly learned (Mowrer, 1960, Chapter 3). But per- 
haps this sort of “passiv " learning, this “baiting of a response with 
secondary reinforcement, 50 that the subject will be disposed to make 
it, is still a sore of trick and does not exemplify “habit formation" in 
] most ty pical form. Perhaps something else, quite 


is in no way exceptional is indicated 


its most general anc 
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dikerat is involved when the correct response is ques initially by the 
subject himself, rather than for him. How can we get evidence on this 


important. issue? 

First of all, it should be noted that both passive and active per- 
formances of a given response characteristically produce certain 
stimuli; and when such a response is, as wc sav, rewarded, ideal con- 
ditions are provided for those stimuli to take on secondary reward 
capacity. In other words, the conditions which are supposedly con- 
ducive to the formation of a "habit," as a putative "bond" between 
some drive stimulus S, and some instrumental response Rj, are precisely 
those which will also give the stimuli produced by R; the capacity 
to act as secondary reinforcers. Thus we arrive at the interesting 
possibility that “habits” are not dependent at all upon a change in the 
resistance or conductivity of S;—R; connections but, rather, involve 
increased conductivity between the stimuli which response Ry produces 
and the phenomenon of secondary reward, or “hope.” In other words, 
just as response inhibition through punishment has been shown to 
depend, not upon a weakening of some S,—R, neural bond, but upon 
the conditioning of fear to the stimuli which R; arouses, so would the 
facilitation of a behavioral tendency which we loosely refer to as 
“habit formation" depend, not upon the strengthening of some Sj—R; 
connection, but upon the conditioning of a different type of “feed- 
back," namely bope, to the stimuli which R; arouses.! 

Perhaps the plausibility of this interpretation of so-called habit 
formation will be increased if we approach it a little differently. If the 
theory which has just been sketched is valid, it would follow that 


"Pam indebted to Dr. Norman Ginsburg for the suggestion (personal. com- 
munication) that this altered way of thinking about "habit? might be applied, 
with equal cogency, to the concept of "instinct." Like habit, instinct. has. been 
commonly regarded as a complex reflex which, however, is innate rather. than 
acquired. Dr. Ginsburg's suggestion is that. perhaps w hat is innate or nativistic 
about an instinct is not certain sensory-motor connections but rather connections 
between certain stimuli (sensations) and certain emotions. Behavior would then 
be modified as a matter of course—but more flexibly than if S—R reflexes were 
directly. involved. Cf. Vinbergen's (1950). statement: “We begin to realize. now 
that the fact that an animal may use various behaviour patterns in order to attain 
one special [instinctive] end does not necessarily mean thar this plastic behaviour 
is not dependent on nervous mechanisms, but that it might also, and certainly 
does, mean that the underlying nervous mechanisms are much more complic ted 
than was expected before” (p. 305). Perhaps the way of conceiving instincts 
which has been suggested by Ginsburg will serve to reduce this complexity. For 
a review of current thinking about the concept of instinct, sec also “Tinbergen 


(1953). 
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Just as responses can be facilitated, i.c., made more likely to occur, 
without their ever having been made by the subject, so can responses be 
less likely to occur, without their being made by 


inhibited, i.e., made 
the subject (cf. Section VI and Chapter 11). In other words, if tradi- 
tional habit theory says that an act can be learned only by occurring 
and being rewarded, it also says that an act can be unlearned or in- 
hibited only by occurring and being followed by punishment. But if 
we now assume that learning can occur, as in the hand-shaking illustra- 
T g, in the active, voluntary sense of that 


tion, without an act occurring 
word, so should we infer that an act can be inhibited likewise without 
as follows. Suppose that a 


occurring, This inference could be tested 
Iv. flexed and that there followed, not 
(either to the leg or to some other part 
It of a few repetitions 


dog's right foreleg was passive 
a reward, but an electric shock 


of the body). What would we expect the resu 
r We would expect that any tendency on the 


his movement voluntarily would be con- 
siderably depressed, rather than augmented—and that the dog would 
thereafter resist having the leg moved passively. This inference follows 
not only from the altered conception of habit which has just been 
at we have now known for at least two 
of punishment. In fact, it was the dis- 
ally a matter of acquired “negative” 
to conjecture that reward similarly 
ar, but of fear reduction and 


of this procedure to be? 
part of the dog to make t 


suggested, but also from wh 
decades about the psychology 
Covery that punishment is basic’ 
(fear) feedback which has led us to € 
Operates to set up a feedback, not of fe 
hope (sce also Mowrer, 1960, Chapter 7). 


Ill. The Neurological Correlates of Learning 


ion, traditional habit theory has been repre- 


For simplicity of exposit 3 Sete à 
D assumption (which it logically 


sented up to this point as involving the 1 it log 
implies) that learning, or habit formation, involves the modification of 
the neural pathway berween some drive stimulus and some instrumental 
response so as to reduce the "synaptic resistance of this pathway and 
increase the likelihood that Sa will be followed by Ri. And the postula- 
tions of Thorndike have been especi lly cited in this connection. How- 
ever, such an imput: ion is not entirely fair and needs to be slightly 
modified. It is true that most of Thorndike's statements sound as if 


they implied this kind of neurologizing, but in 1931 Thorndike made 


an explicit disclaimer on t 


his score. He said: 


nection-system which | have described is more 


l hope that the sort of conn \ ! \ 
ainst which configurationists . . . direct. their 


acceptable than the kind ag: 
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criticisms—criticisms from which I have profited and with which I often 
agree . . . (p. 130). 

The connectionist welcomes the factual criticisms of an over-simplified 
conduction system. The connectionist indeed realizes the difficulties of ex- 
plaining human nature as a system of connections between neurons . . . 
The word connection has been used without prejudice concerning what 
physiological event or condition parallels it. It is so far simply an expression 
of the probability that a certain Sw ill be followed by a certain R, bond, or 
link or relation or tendency, or any still more colorless word may be put 
in its place (p. 7). ? 


For Thorndike, at least as of 1931, habit was therefore simply a 
word for the increased probability of a response's recurrence when its 
prior occurrence has been followed by reward. But there is ample 
evidence that others have taken the terms "connection," "bond," and 


“association” more literally, i.e., have, quite reasonably, thought of 
them in specifically neurological terms. For example, Wolpe, in a 
paper published in 1950, in a section headed “Reinforcement as a 


Neural Process,” says: 


If it is presumed that the stimulus-response sequences set up by learning 
depend, just as maturationally established sequences do, upon the develop- 
ment of functional neural connections, then the process of establishing 
such connections is the essence of what is called reinforcement, and an in- 
dividual event that has the effect of initiating or strengthening a functional 
neural connection is a reinforcement (pp. 19-20). 


And two years later, in a second article along similar lines, Wolpe 
(1952) re-afhrmed this view by saving: "s.s learning depends upon 
the formation or strengthening of synaptic connections at points of 
anatomical apposition of neurones, a synapse being a functional point 
of contact between neurones” (p. 194). 

Hull. also, was a connectionist in the neurological sense of the word; 
but Spence, though in some respects a follower of Hull, has himself 
been careful to avoid making any commitment on this score and has 
said of Hull's neurological “entanglements”: 


Just why Hull, after formulating his mathematical theory of babit, found 
it necessary to elaborate a neurophysiological model of receptor-effector 
connection, has always been a puzzle to mc. Actually he does little more 
than identify or coordinate this concept of receptor-effector. connection 
with his mathematical construct of habit. | doubt whether a single one of 
the deductions with respect to learning in his Principles of Behavior would 
be lost or changed in any way if it were eliminated. These implications fol- 
low exclusiv ely from his mathematical theorizing and not at all from the 
superfluous physiological model. The same is true for Thorndike’s theory 
about the alteration of sy naptic conductances” (Spence, 1950, pp- 163-164). 
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Birch & Bitterman (1949) likewise observe: “anyone who has read 
akes use of supporting physiological data.” To 
comment on the tendency of 


regarding the neurological 


Hull knows that he m 
which they then add this pertinent 
others to avoid making any commitment 
substructure of learning: 

But irrespective of Hull's practice, the present writers hold that since 
the material substratum for learning is the nervous system, a fruitful theory 
is one that vields inferences of a neurological kind, w hich in turn yield 
testable hypotheses crucial for the theory. A non-reductive approach in- 


rather than theoretical strength (pp- 302-303). 


dicates theoretical weakne 
For an extreme nonreductivist position, the reader is referred to 
B. F. Skinner (1953b). who, for example, says: 


vost of the concepts which have 


Frequency of response is emphasized by n A: | 
foreshadowed an explicit recognition of the probability of a datum. Àn 
a "habit" to the extent that à. certain form of behavior 


Organism possesses : H i 
pecial frequency—attributable to events in the history 


is observed with a s$ 
of the individual (p. 69)- 

The basic datum in analysis of 
probability (p. 78)- 


behavior has the status of a [mathematical] 


And Spence (19512) takes the position that: 


at is, rewards) that are known to result 
of the response, the theorist holding 
nes that “rewards” operate in some 
al connection of the response 


, In the case of those events (th 
in an increased likelihood of occurrence 
to a general reinforcement theory assumes. 
unspecified manner to strengthen the functiona € t 
to its eliciting stimulus. So long as the properties of this class of events (re- 
wards) are BOE specified, other than that they lead to increased respons 

strength, a theorist may be said to be supporting a general 


and, unfor- 
tunately, thoroughly circular | 


reinforcement interpretation (p. 245). 


It is easy to see why some habit theorists have, in the past, been 
reluctant to take a definite position concerning the neurological cor- 
relates of learning; To assume that a “habit” involves the lowering of 
synaptic padietanec between Su and Ri, where Ri consists of activity 
Which would ordinarily be termed “voluntary, Is to imply a rigidity 


and fixity of so-called “habitual” behavior which goes contrary to 


Common experience anc 
! " Py 
ask where rev 


1 observation. 

Now we must ised two-factor theory stands on chis 
question of learning and neurology. 
theory assumes that learning does indeed involve “the formation or 

À stic connections at points of anatomical ap- 


strengthening of synap 
position of neurones” (Wolpe's statement). But there is an important 
which connections. Traditional habit theory 


The answer is quite simple: This 


difference here as to 
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has stated or at least strongly implied that jennie involves a strength- 
ening of synapses between neurones connecting some drive and some 


behavioral act, whereas revised two-factor theory assumes that so-called 
habit formation involves a strengthening of synapses between the 
neurones connecting stimuli produced by some behavioral act and 
the emotion of hope and that punishment involves a similar condition- 
ing of fear. In other words, the assumption here is that learning is rc- 
lated, exclusively, to the connections involved in the informational 
feedback from a response or response "intention" (i.e, a partial 

perhaps symbolic occurrence of the response), and that it involves 
no change in what may be called the "executive" (brain-to-muscle) 
pathw: ays in the nervous system (see Mowrer, 1960, Chapter Th. Ly 
this way we avoid the dilemmas which have dogged habit theory for 
these many years: such a view leaves behavior—even so-called habitual 
behavior—free, flexible, and uncommitted while at the same time it 
does not hedge on the question of the neurological correlates of learn- 


ing. 

The issues here at stake can be summarized as follows. The S—R 
bond conception of habit formation has to assume (a) that learning 
consists of an increase or a decrease in the conductivity of the neural 
connection between drive and behavior or (b) that no such neuro- 
logical modification occurs and that the change in response probability 
is mediated in some other (usually unspecified) way. Neither alternative 
is logically attractive. The position here suggested is that learning, 
clearly and unambiguously, involves the strengthening or weakening of 
neural bonds (C hapter 11) but that these bonds are ‘between formerly 
neutral stimuli and "conditioned responses" (notably the emotions; 
but see also Mowrer, 1960, Chapter 5). Learning thus alters what the 
individual wants and wants to do (and docs) but not what he can do. 


The capacity for executing behavior is assumed to be innately given 
and (probably) not alterable through learning. Some of the factual 
support for this statement, summarized by Sperry (1951), runs as 
follows: 


Adaptation in the timing of the central discharges is not achieved bv the 
learning process. If limbs | of mature amphibia | are transplanted to dorsal 
positions such that their movements are of no value to the animal, the central- 
peripheral adjustments occur in the usual svstematic manner. This is true 
even when the limbs are transplanted to the contralateral side and. reversed 
in such a way that the movement of the limbs rends to push the animal back- 
ward when it attempts to go forward and vice versa. Similar effects arc 
obtained when the limb transplantations are made in prefunctional stages, 
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show ing that these relations are patterned initially through developmental 
forces and not through any kind of functional adjustment. Furthermore 
it has been shown that these motor patterns develop in the same systematic 
Way in the absence of sensory innervation and that they persist after de- 
cerebration and cord transection down to levels of the cord just rostral to 
the limb segments. The possibility that learning or any type of functional 
adaptation might be responsible has thus been excluded. The motor neurons 
must somehow be distinguished from one another in the centers according 


to the muscles they innervate. . . ." (pp. 2577258). 


goes on to point out that: 


However, Spe 
this kind occurs readily in the limbs of amphibian 
v during and shortly after metamorphosis and 
evidently does not occur at all in full-grown anurans. In the phylogenetically 
More ancient oculo-motor system of amphibians the capacity for reorganiza- 
tion seems to be lost at a very early larval stage in both anurans and urodeles. 
-.. In the rat the relations of the limb motor neurons has been. found 
to be already irreversible at, or shortly after birth. The newborn oppossum 
has vet to be investigated in this respect. Where adjustment fails to occur, 
it may be attributed to the fact that cither the motor neurons in postem- 
bryonic and postlarval stages are no longer subject to respecification, or 
else that such respecification no longer causes a breakdown in the existing 


Synapses” (p. 260). 


Motor reorganization of 
larvae. It goes more slow! 


And Carmichael (1946), more than a decade aga oaee an 
Point that in so-called trial-and-error learning, the response has to be 
made before “learning” can begin. 1 herefore learning (as already 
indicated) must be more a matter of control, of coming to want or 
articular response, than of capacity. It is only 


not want to make a pi : ; 
it seems, that is altered by 


the relative attractiveness of the response, 
tion is unequivocally assumed to be neurolog- 


as we shall see in later chapters, calls 
about psychology in general; but 
these readjustments seem not to be in any sense stultifying but instead 
liberating and creative (see also Mowrer, 1960, especially Chapter 7). 

Since the above was written, a popar Eg Sie Ponisi neúrophysinlogist, 
J. Konorski (1958), has appeared which sighed additional light Spon 
the issues here under discussion. This writer s thesis is briefly this: C. S. 
Sherrington (1906) provided a satisfactory conceptual model for the 
Operation of the spinal cord and the reflexes which it mediates, but no 
one has given us a satisfactory model for the operation of the “higher 
E viz. the brain. Here is w here learning manifestly 
ise the reflex model has proven seriously inadequate 
there has been a widespread tendency on the part 


learning; and this altcra 


ically based. This point of view, 
for some changes in our thinking 


s; and becat 
1n this connection, 
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of the students of learning to abjure and reject neurology altogether, 
with the results that only formal or mathematical statements, in the 
last analysis, remain. Konorski says: 


According to my own view, purely formal behaviourism is in the long 
run sterile. In fact, if we make the decision to refuse to accept psychological 
interpretations of animal behaviour, we are bound sooner or later to sub- 
stitute physiological ones; otherwise we are doomed to remain on a purely 
empirical level without any possibility of creating a causal system of the facts 
observed. Therefore I consider that the gradual “physiologisation” of be- 
haviouristic psychology is an “historical necessity," and we are now wit- 
nessing the acceleration of this very process (p. 1107). 

Pavlov's theory of cortical processes could not play the part of a frame- 
work in the physiology of the brain, did Sherrington's concepts—dc- 
veloped at about the same time—as regards the functioning of the spinal 
cord; thus, the physiology of higher nervous activity failed to become 
“assimilated” by neurophysiology as an important new branch. This in- 
volved two important consequences. On the one hand, the behaviourists, 
though representing a trend quite close to that of the physiology of higher 
nervous activity by which they were greatly intluenced, declined to assimilate 
Pavlov's theory of cortical processes (even in spite of their not infrequent 
good will), and turned away altogether from the physiological approach to 
the problems of behaviour, going the way of formalism. On the other hand, 
contemporary neurophysiology became mainly "analytical" in character, deal- 
chitecture of the connections between particular 
iring but little for their functional significance (p. 


ing primarily with the 
parts of the brain, and ca 
IIIO). 

The speculative brain physiology of Pavlov has thus collapsed under 
its own dead weight; and by its own inherent logic and evidence, be- 
havioristic research has shown the inadequacy of the “brainless 
(spinal) model of Sherrington. So where do we stand today? The two 


positive facts in the situation are these: (a) Behavioristic researches 
have led to a new and different (nonreflexive) conception of behavior 
control and selection; and (b) neurophysiology, with its numerous 
recent improvements in technique, has given us a far clearer picture 
of the course of pathways in the brain than had ever before existed. 
It seems likely that these two facts will soon be brought together in 
1 


a fruitful. creative way. In later chapters we shall. indeed, see indi 
tions that. just such a development is, as Konorski intimates, already in 
progress (sce also Chapter 6, Section IX). 


IV. An Earlier Statement of the Hypothesis 


The intriguing, but also rather disconcerting. possibility that “habit 
is simply a matter of secondary-reinforcement feedback was first ex- 
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plored by the present author (19532), in an unpublished paper with the 
title: “Is ‘Habit Streneth’ Merely Secondary Reinforcement?” ? The 
idea has also been considered, in a preliminary way, in two other papers 
In the first of the three papers cited, the 
ations were developed as follows: 


(Mowrer, 1953b, 1936). 
basic notion and some of its implic 


Among the graduate students in our department, there is a bit of com- 
a ? If an instructor asks, “What is wrong with th 
“the correct answer is: “The experimentor 
orcement. " And as we become 


mon folklore to this etfe 
(or that) learning experiment?” rec 
did not properly control ‘secondary reinf 6 vi we Deci 
practiced in thinking about this phenomenon, we can see its subtle and in- 
1 maze and memory drum. So far researchers, 


have regarded secondary reinforcement as à 
frustrates their efforts to get at the "rcal 
thing," namely the. basic. process of learning whereby habit pm inei E 
S—R bonding, occurs |see also Chapter 11]. Here we shall explore the 
possibility thie secondary reinforcement 15 itself the real thing, with "habit 


as ` i srivative: 

Let us approach the tatter as follows. Recently we have been preoc- 
cupied with a number of animal experiments on west Poo reer tent 
in one of which the basic procedure is a5 follows. 1 MU. s ie e 
sented, twenty times per day on four CIERRE E = ea $ S d M 
before the subject (a rat) is fed. On the fifth i ills b "E des à D RR 
different. situation, with a “Skinner bar availa n { i dine de neh 
produces the blinking light—but no food. As might x egi DU Sinnen 
1938; Jenkins, 1950; also Chapters 4 and iol, paged pene ms ay E 
press the bar, solely on the basis of the second um) Fi ; dox o ue dne 
by the blinking light. Of course the performance o "e b S fs I ter i 
ceases because the secondary reinforcement EATIN EMIS Cok CHE LOK A MAE 


sidious operation in every 
More commonly than not, 
process which ubiquitously 


least, in old—and in- 


T Š 5 ENS esed, for a time 
* Sonierimes new ideas: have to. be-expressecs ; j : Ae 
appropriate terminology. For example, there i5 an implied contradiction in say- 
ing that a response has 
stimuli which are produced by 
reinforcing capacity. Capacity to. 
enee, the answer would be: reinforce © ^ 3 aue i 
bond which mediates the respor But if we rans eo conception of 
habit, then the notion of "reinforcement" must be reconsidered. As will be shown 
abit, then the no 


secti tjs jr is from one point of view more 
: : ction. IV), it is fron 1 
n Chapter & (see espec lly Sect 


ace ct 4 ivation". rather than 
Accurate. to speak here of “motivation Ti n a 
| reinforcement 1s still valid, as refe 
stimulus which has acquired so-called. secondary 


"habit strength? in proportion to the extent to w hich 
i the occurrence thereof have acquired secondary 
“reinforce” what? In the older frame of refer- 
thicken,” “open™) the stimulus-response 


ase. 


of "reinforcement; but in 


ring to an agent which 
another sense, the tern 


produces new learning. Any l m equi 
d in addition to the "morati 


tv to act as the “unconditioned” (condition- 

ine) sciniulüs in higher-order conditioning. In this book, we shall continue to use 
x imulus in higher- aoe fae ak ^ H 

a i inforcement™ with these qualifications in mind; but even- 

re < : 

will have to be found which more precisely reflects. the 


(behav ior-releasing ) 


reinforcement capacity has. 
function just mentioned, the capa: 


the term “secondary 

tually a terminology , 3 

Teese V gut wieniec understanding (see also Chapter 12, Sections HI 
state o 


and IV), 
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we have, purely on the basis of secondary reinforcement, a fine little “habit.” 

This simple demonstration suggests the following line of thought. By 
having associated the blinking light with food (ie, with hunger-drive ré- 
duction), we caused it to acquire secondary reinforcing properties, and 
this was enough to give our subject the “habit” of pressing a bar which 
produced the blinking light | see also Section V |. n 

But this, we may be told, was a special, rather tricky habit, not a real 
one. Let us, then, set up a “real” one. To do this we put a naive hungry 
rat into an experimental situation with the Skinner bar arranged so that it 
produces, not a blinking light, not a mere promise of food, but food itself, 
ic. so-called primary reinforcement. Again we get a bar-pressing habit— 
this time a "real" one. But wherein does it differ, basically, from the “habit” 
first described? In both cases we see habit strength manifested by resistance 
to extinction, i.c, by a tendency on the part of the rat to perform the re- 
sponse after food ceases to be forthcoming—and, one may suppose, for the 
same reason: In the one case the animal gets secondary reinforcement from 
the blinking light which bar-de ion produces, while in the other case 
the secondary reinforcement derives from the proprioceptive, tactile, visual, 
and other stimuli inberently associated with tbe response. Wow did these 
latter stimuli acquire secondary reinforcing properties? Presumably in the 
same way the blinking light did—by having occurred contiguously with the 
reinforcement provided by food and hunger reduction! š : 

Recently the writer was trying to explain this line of reasoning to a col- 
league at another university, and when he had finished the colleague said: 
“But the rat, in the first case, did have to learn to press the bar; he did 
not just start doing it all of a sudden.” 

While acknowledging the reality of secondary reinforcement, this ps 
chologist was so steeped in the tradition that habit formation represents a 
growth in the neural connection between drive and response that he could 
not understand what the writer was saying. And, more interesting still, the 
writer himself. was so unaccustomed to this other way of thinking that it 
was not until several hours later that he realized (too late, alas, for a direct 
rejoinder!) how he should have replied. What he should have said (ob- 
viously, now ) is this: "Yes, the rat did have to learn to press the bar. But 
there is this difference. You are assuming that this learning, this habit for- 
mation, consisted of the rats learning bow to press the bar. I am sug- 
gesting rather that the habit formation consisted of the rat's learning to 
want to. press the bar.” Pretty clearly the rat, when put into the situation 
with the Skinner bar, already bad the necessary know-how, or skill, to 
perform this simple act, in a dozen different ways. But at first the rat just 
didn't particularly want to. Why should it? It pr erred to “explore” [see 
Chapter 6|. In the course of its exploration, the rat ev entually got around 
to pressing the bar—and when it did so, something special happened: the 
blinking light came on. This one, initial performance of the bar-pressing 
response shows that the rat already “knows bow.” The result was that the 
various stimuli inherently associated with the bar pressing now began to 


5 "D raditional experiments in learning have usually been concerned with changes 
in the character of behavior. The organism learns bow to do something, it ac- 


quires new behavior" (B. F. Skinner, 1953. p. 70). Cf. the following excerpts from 
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take on secondary reinforcing powers —and, as this occurred, the bar press- 


ing "habit" became manifest. 

|n other words, having been once paired with the blinking light, thc 
stimuli indigenous to the act themselves acquire some "reinforcing" capacity 
and thus increase the likelihood of the act's being repeated. And with each 
repetition of the act, the indigenous stimuli will be temporally associated 
with the blinking light and will acquire further reinforcing properties. This 
transfer of reinforcement from the extrinsic stimulus (blinking light) to the 
Intrinsic (proprioceptiv e, tactual, visual and other) stimuli continues, then, 
With each occurrence of the bar-pressing, light-producing response, until 
both the extrinsic and intrinsic stimuli lose, through extinction, their rein- 
hich point we say the "habit" is gone. 

What we thus arrive at is a conception of “habit formation” which is based 
Upon sign-learning, or conditioning, and which, as we shall later sce, is not 
encumbered by the difficulties which beset both the stimulus-response bond 
conception of habit and the simple conditioned-reflex approach to behavior 


modification (Mowrer, 1953s PP: 5-10). 


forcing potency—at w 


V. Supporting Evidence: 

(a) The Bugelski-Miles Experiments 
of habit which has just been elaborated in 
rimental facts—some of them 
atly into place. In this section 


With the conception 
mind, it is surprising how man 
cable—now drop ne 


number of empirical 
esistance to extinction, which lend them- 


M expe 
not previously. expli 
We shall review a 
Strength," as measured by 7 


findings concerning “habit 


“Thorndike points out that in maze experiments 


able to go quickly to a certain place and 
of learning to wish to go there 1932b, p. 458). Tolman . . . dis- 
tinguishes betw cen acquiring field expectancies and positiv c cathexes; the corre- 
Sponding distinction in maze learning would be between field expectancies and 
‘subgoal cathexes. A similar distinction is conceivable between learning any 
response and acquiring a need to perform it. The most economical solution, and 

E is to subsume "learning to W ish’ and "learning to do’ 
171) [see Chapter 81. 


under a single inclusive principle" (p. 3 d 
is in order here. We have spoken above of response- 


rcinforcing capacity by virtue of their 
lso possessed this capacity. Strictly 


a paper I 
there is a 


Seward (1950): 
mixture of learnir 


ig to be 
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i 
A note on terminology 
Produced stimuli taking on secondary 


association with the blinking light, which a A 
Speaking, one should. perhaps refer to secondary. reinforcement only with respect 


to stimuli which have been associated with primary reinforcement, to tertiary 
reinforcement with respect to stimuli which h A 
reinforcement, etc. But it is à grow ing practice, and a well-justificd onc, to 
speak only of primary reinforcement and secondary reinforcement, the former 
referring toa reduction in ry drive and the latter referring to 3 
m č or emotion, without reference to how far removed, in 
it is from primary reinforcement. In this discuss 


ave been associated with secondary 
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reduction 


a prima 


secondary drive 
terms of stimulus pairings. 
We shall follow the latter usage- 


sion, 
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selves particularly well to interpretation within this new theoretical 
framework. 

Reference has already been made (Chapters 4 and 5) to Bugelski's 
discovery that a Skinner bar-pressing habit “extinguishes” more 
rapidly if its occurrence is not accompanied, during the extinction 
period, by stimuli which accompanied it during the acquisition period. 
Because this discovery is so basic to the present analysis, it will be 
useful to read a part of Bugelski's own account of this now twenty- 
vear-old experiment, as given in his 1938 paper, "Extinction with and 
without Sub-goal Reinforcement.” It begins: 


In recent theoretical papers Hull (1932, 1937) has posited sub-goal mech- 
anisms as of major importance in serial behavior acts. White (1936) has 
based his completion hypothesis on the reality of the anticipatory goal re- 
action functioning as a behavior-directing stimulus; he adds, however, that 
as not vet been widely accepted or supported by crucial experiments." 
The Skinner (1932) bar-pressing technique, as currently used in the study 
of learning problems, provides an example of such a sub-goal mechanism 
operating in a concrete laboratory situation. 

A common practice in the preliminary training of rats by the Skinner 
method is to deliver to the animal a series of uniform pellets of food. Upon 
the closing of an electric circuit one pellet of food is delivered to the animal 
automatically by an electro-magnetic vending device placed outside the 
partially sound-shielded experimental box. The falling of the hard pellet 
down the chute into the metal food cup produces a distinct rattle and the 
action of the magnet is transmitted mechanically through the walls of the 
box as a click. Under these conditions rats very quickly acquire the habit 
of running to the cup to get the pellet as soon as the sounds occur. After 
is trained to operate the 


this associative connection has been set up the 
food-delivery mechanism himself by pressing downward on a short hori- 
zontal bar. The preliminary training to the rattle and click. presumably 
hastens the learning of the bar habit by insuring that whenever the animal 
accidentally presses the bar this action shall be followed very closely by the 
reinforcement associated with the eating of the food. Such close temporal 
relationship between act and reinforcement has long been known as a con- 
dition favorable for rapid learning. 

Once the bar-pressing habit has been set up. it is plausible to assume that 
in some sense the click becomes a significant subordinate goal, the attainment 
of which, according to White's hypothesis (1936), should have reinforcing 
properties. On the other hand, the absence of an accustomed reinforcing 
state of affairs normally tends to produce experimental extinction. On these 
assumptions, then, if experimental extinction of the bar habit is carried out 
in such a way that each act of the animal is followed by the magnetic click 
bur no food, it is to be expected that extinction would take place at a slower 
rate than it would if performed in the same way except w ithout the magnetic 
click. The present experiment was designed to test this hypothesis (pp. 121- 


i22). 
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There then follows the conventional description of subjects, ap- 


paratus, procedure, and results—after which there is this summary: 


a bar to obtain a food reward. 


Sixty-four albino rats were trained to press ol i 
Operation of the bar wa :companied by a sharp distinct click. After the 
f this habit they were divided into 


animals had received 30 reinforcements 0 
two groups and extinguished to a criterion of five minutes without a response. 
Half the animals were extinguished under the original learning conditions 
which included a sharp click. The other animals did not hear the click during 
extinction. 3 IN 

The results indicate that the animals who heard the click during extinction 
responded to the bar more often than did the animals who did not hear the 
click. | An average of 77.60 extinction responses as compared with an average 
of 54.06.] The difference between the mcan number of responses was over 
30° in favor of the rats who heard the click. These results correspond with 
deductions drawn in advance of the experimental work. 

After the first extinction the rats Were divided into groups of 16; 32 rats 
now extinguished under reversed. conditions of "click" or "no click." The 


remaining 32 rats extinguished a sccond time under the original conditions. 
> t d; those animals who heard the click made 


The same general results followe t i Cao 
more responses than those animals who did not hear the click. The results 
again corresponded to theoretical deductions drawn in advance of the experi- 
mental work. 

On the basi 


; of the results it appears fairly certain that the presence of the 
click during extinction v 1 partial or "sub-goal to the animals and that its 
5 sulting from the absence of the 


absence added more frustration to that re i 
food reward [see Chapter 11 |. These findings offer an experimental demon- 


stration of the sub-goal principle advanced to explain di den features of 
serial acts and the satisfying nature of an anticipatory goal response (pp: 


132-133). 


for present purposes. First of all, 
method employed by Skinner for quickly setting 
up the bar-pressing habit, it provides à nicc illustration of the point 
that learning is not necessarily dependent upon doing. 1 he click of the 
delivery mechanism and the sound of the pellet tumbling into the food 
trough are two stimuli which will follow the response one wishes the 
c when the bar is made available. They, in other 
rrelated with respect to the bar-pressing 
aining procedure does, therefore, is to give 
reinforcement potency bya special, 
Then, when an opportunity is pro- 
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reinforcement of the connection between the stimulus of seeing the 
bar and the response of pressing it, instead of allowing the strengthening 
of this “bond” to depend entirely upon the necessarily somewhat 
delayed primary reinforcement provided by ingestion of the food. 
The alternative interpretation which our particular conception of 
"habit" suggests is that, by “baiting” some of the stimuli which are 
naturally associated with bar pressing, the Skinner method is, in effect, 
a sort of latent- learning procedure (Mowrer, 1960, Chapter 2) in 
which the bar-pressing ‘habit is partially learned in advance of ever 
occurring. Just as the boy who flexes his dog's right forepaw and 
then gives the dog food is predisposing the dog to make this response 
himself, so does the Skinner method of "pretraining" tend to create 
a habit without the subjects engaging in trial-and-error behavior or 
otherwise exerting himself. The present assumption is that if all the 
stimuli normally associated with the occurrence of a particular re- 
sponse could be produced thus “artificially,” i.e, without the occur- 
rence of that response, a "habit" could be established which would 
have just as much "strength" as if the response had itself occurred and 
been rewarded. 

What, now, may we say of Bugelski's study of extinction, with and 
without the noise normally made by the operation of the (empty) 
delivery mechanism? The more conventional interpretation of the 
Bugelski findings has been that the bar-pressing habit, properly speak- 
ing, consisted of an S,—R;, neural connection which, when primary 
reinforcement was withdrawn, staved alive and active. (experimental 
group) longer than it otherwise would (control group) because of the 
strengthening, sustaining effect of the secondary reinforcement pro- 
vided by the sound of the food-deliv ery mechanism. According to re- 
vised two-factor learning theory, this finding i is taken as showing that 
instead of the "strength" of a "habit" being maintained or reinforced 
by stimuli with so-called secondary reinforcement capacity, habit 
strength is stimuli with secondary reinforcement capacity and that a 
“habit” is strong or weak, not as a function of the S,—R; “bond,” but 
as a function of hoz many of the stimuli produced by a response 


possess bow much of this so-called reinforcing potential. If this po- 
tential is extinguished or if the stimuli which a given response usually 
arouses arc, by whatever means, prevented from occurring, it is assumed 
that the “habit” will be obliterated forthwith, whatever the state of 
the S,—R, connection. 

Recently an experiment was reported by Dr. Raymond C. Miles 
(1956) W hich further clarifies and empirically: supports this type of 
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thinking, In the first part of this study, laboratory rats learned to press 
a Skinner bar as a means of obtaining food and were then subjected 
to extinction, with and without two external sources of secondary 
reinforcement. During the acquisition or training period, cach bar 
depression was accompanied by a flash of light and the click made by 
the food-delivery mechanism. In the case of some animals, the bar- 
pressing habit was extinguished with the light and click occurring at 
i (but no food); and in the case of other 


cach depression of the bar 
neuished with the light and the click both 


animals, the habit was extir 
"disconnected" (and again no food). 

Up to this point, our description of the Miles experiment. docs 
not differentiate it, in principle, from the experiment by Bugelsk 
previously. discussed. The distinctiveness of the Miles experiment lies, 
first of all, in the fact that the procedure was replicated with wide 
variation in the number of food-reinforeed bar-pre ing responses 
which were permitted during the period of training, prior to extinction. 
In all, 240 animals were used in this part of the study; and they were 
randomly divided into six groups. of 4o animals cach. "These six groups 
Were given an opportunity to press the bar and receive food the follow- 
ing number of times: 0, 10. 20. 49 go. and 160. After this training, 
cach of the six groups of 40 animals was divided, again randomly, into 
two subgroups, of 20 animals each; and these twelve subgroups were 
tested for resistance to extinction under the two conditions already 
mentioned. The results are shown graphically in Fig. 7-1. Here it will 
be seen that the decrement produced in resistance to extinction by 
withholding the light and click was re 


latively. constant for the various 
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groups which received different amounts of original training in making 
the bar response. The stability of the ratio between resistance to extinc- 
tion with and without the two external secondary reinforcers, regard- 
less of amount of original training, is shown in Fig. 7-2. 


1.00 
Fig. 7-2. Values derived by 
dividing scores for No- 
Secondary - Reinforcement 
Groups by corresponding 
scores for Secondary-Rein- 
forcement Groups, as shown 
in Fig. 7-1. As will be seen, 

y 


o 

œ 

o 
. 


Responses 
o 
o 
eo 


o 
D 
o 


the resulting quotient is ve 


0 10 20 40 80 160 nearly a. constant. 


Habit strength 


What do these results mean? Miles suggests that they may offer a 
basis for partialing out the extent to which the resistance to extinction 
shown by the bar-pressing response was due to external, secondary 
reinforcers and the extent to which it represented habit strength, 
properly speaking. But there is a suggestive fact to be noted here: It 
turns out that both habit and secondary reinforcement develop, with 
progressive reinforcement, according to what appears to be exactly 
the same function, This is shown, first of all, by the curves in Figs. 
. It is further indicated by a comparison of the lower curve 


7-1 and 


in Fig. 7-1 and the curve in Fig. 7-3. Miles constructed the latter by 


taking the successive differences in amount of resistance to extinction 
for the various groups shown in Fig. 7-1: and plotting these differences 
independently. He reasoned that these differences ought to show the 
growth of secondary reinforcement, as opposed to habit strength 
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The shape of the growth function for 
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Tn. short, then, the Miles investigation shows (a) that “habit” and 
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i.e., hunger. Here, it seems, is about as compelling evidence, of at least 
an indirect nature, as one could ask for the hypothesis that “habit 
strength” and “secondary reinforcement” are one and the same thing, 
the main difference between them being that when secondary rein- 
forcement is connected to internal, response- produced stimuli we call 
it—for no very apparent reason—"habit." 
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From the text of the Miles paper, it is clear that the author himself, 
at the time this paper was written, was of two minds as to the meaning 
of his very systematic and empirically beautiful findings. He was, on 
the one hand, inclined to differentiate between what may be called 
inherent habit strength, i.e., the established “thickness” of the S,—R, 
"bond," and the capacity which external stimuli acquire to impart ad- 
ditional strength to such a habit or bond through the action of sec- 
ondary reinforcement. At the same time, Miles gives careful con- 
sideration to the possibility that so-called habit strength is exactly 
equal to the so- called secondary reinforcing capacity processed by 
all Tesponse- correlated. stimuli, be they external or internal as far as 
the organism is concerned. Certainly the results obtained by Miles and 
by other investigators whom he cites suggest that habits lose their 
“Strength” pari passu as responsc- produced. stimuli, which have been 


25 
20 m Fig. 7-6. Curve showing the 
ih x < increasing discrepancy be- 
2 15 k - tween corresponding values 
8 shown in Fig. 7-4. as a func- 
$ 10 e tion of hunger intensity. The 
/ curve, like that shown in Fig. 
5 . has much the same shape 
0 as the typical learning curve. 
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previously associated with primary reinforcement, are excluded (or 
. Present theory holds that a response which arouses 


“extinguished” ) 
roperties possesses, ergo, 70 


no stimuli with secondary reinforcing p 
“habit strength." 


VI. Supporting Evidence: 
(b) Further Indications that "Habit" Is 
"Secondary Reinforcement" 


vo studies cited in the preceding 


In addition to the results of the tv 
findings, both formal and 


section, there are many other laboratory 
Informal, which support the view that "habit" is nothing more nor 


ess than the secondary reinforcement, so-called, which has become 
attached to response-correlated stimulation. For example, in a personal 
I. O'Kelly has reported the following 


communication, Professor L. 
xperiment conducted for a different 


Observation. In an exploratory € 
ivated by thirst and housed in indi- 


Durpose, several white rats, moti 

vidual compartments, learned to press a bar and cause a drinking tube 
to become briefly accessible. Extinction was then conducted under 
two different conditions: (3) bar pressing caused the motor which 


ad previously delivered the drinking tube to make its customary 
sound but without producing the tube; and (b) bar pressing caused 
the motor to sound and to deliver the tube but with no water in it. 


Professor O'Kelly reports that animals with which the second of these 
i “about 12 times as resistant to extinction" 


s there was virtually “no over-lap 
oups.” Although there are perhaps 
ting this observation, it is clearly 
c pletely the stimulus conse- 
iling under 
cater will 


Procedures was used were 
as were the other animals. He $ 
at all in the scores for the two $! 
ays of interpre 
ew that the more com 
quences of a nonrew arded re approximates those prev 
the antecedent conditions of primary reinforcement, the gi 
be the resistance to extinction. This observation. and interpretation 
t with the facts previously cited (Chapter 6) in 
of sham cating. The O'Kelly pro- 
at least quasi-sham-drinking since 
as of drinking—lapping. etc.— 


other acceptable w 
Consistent with the vi 


sponse 


thereof are congruen 
Connection with the discussion 
an instance of 


cedure is, in effect. 
all the motior 


the subjects went through 
In the tube's presence. 

If the foregoing hypothe 
Which calls for d given amount 0 
Would be more resistant to extin 


call for this, rather than some other, t 
tion is that the “habit” thus acquired consists largely of the secondary 


sis is sound, it would follow that a response 
f muscular effort during acquisition 
ction if that response continued to 
amount of effort. The assump- 
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reinforcement which has become attached to the proprioceptive feed- 
back from (or “feel” of) a given amount of etfortfulness and that if 
the response is made either wore or less etfortful during extinction, 
its habit strength will be forthwith diminished. Although no attempt 
has apparently as yet been made to test this inference, directly and 
specifically, results reported by Stanley & Aamodt (1954), in a paper 
entitled “Force of Responding During Extinction as a Function of Force 
Requirement During Conditioning," arc consistent with expectations 
generated by this inference. In a personal communication, Dr. Stanlev 
expresses the opinion that although, in the article cited, the obtained 
results were interpreted. somewhat differently, they are nevertheless 
consistent with the notion of “proprioceptive scanning, as mentioned 
in your recent Review article" (Mowrer, 1956; cf. also Chapter 11). 
And Wyckoff (1959). in a recent. paper, says: 


The profound effects w hich may result from subtle similarities and ditfer- 
ences in stimulus conditions between acquisition and extinction have been 
brought to light in the context of studies on the influence of intermittent rein- 
forcement (Crum, Brown, & Bitterman, 1951; Schoenfeld, Antonitis, & Bersh, 
1950; Spence, 1947; and Tyler, Weinstock, & Amsel, 1957) [see Lawson & 
Brownstein, 1957; also Chapter 12] (p. 2). 

Also relevant, and even more directly so, is a study by Melching 
(1954), the results of which are summarized in Table 7-2. The pro- 
cedure was as follows: 


Six groups of to rats cach were conditioned to press a bar for food. For one 
pair of groups [C and D| a buzzer sounded on half the occasions the bar 
was pressed. For a second pair of groups {A and B] a buzzer sounded cach 
time the bar was pressed. For the final pair of groups | F and F] the buzzer did 
not sound at all during conditioning. In extinction, the buzzer sounded 
sed the bar. The buzzer did not sound 


when one group in cach pair pre 
when the other group in cach pair pressed the bar (p. 373)- 


TABLE 7-2 


Total Extinction Scores of Groups Conditioned under Three 
Schedules of Buzzer Presentation and Extinguished 
under Two Schedules of Buzzer Presentation 
(Melching, 1954) 


Frequency of Buzzer Frequency of Buzzer Number of 
Group in Conditioning (“7 ) in Extinction (4: ) Extinction Responses 

A 100 100 755 
B 100 o 574 
G 50 100 719 
D 50 o 727 
E o 100 402 

637 
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As will be seen in Table 7-2. if animals received the buzzer dur- 
ing cach and every acquisition bar press, they were more resistant to 
extinction if the buzzer continued to sound (Group A vs. Group B); 
whereas, if animals received no buzzer during acquisition bar pressing, 
f the buzzer did vot sound dur- 


they were more resistant to extinction i 
ing extinction (Group E vs. Group F). In other words it was found 
resistant to extinction Ge. has greater “habit 
it continues to produce exactly the 


samme rather than a different pattern of stimulation. This finding has 
been independently confirmed by Hurwitz & Cutts (1957) and is in 
line with the view that habit is mercly the secondary reinforcement 
which has been conditioned to the stimuli associated with the response 
during acquisition. (The results reported by Melching for Groups C 
and D lend themselves to no simple interpretation—cí. Chapter 12— 


and will not be discussed here.) 


Thus far in this (and the preece t : 
g, during extinction, previously rclevant— 


only the effect of withholding ENIS l 
or adding previously irrelevant—stimull. And the findings are strictly 
in accord with the hypothesis under examination, Even more striking 

M not only by the procedures 


is the finding that a habit can be depressed, cedures 
h Iso by what may be termed the “pre-extinction 


just mentioned, but a , 4 A 
of relevant stimuli. At the outset of the preceding section, a passage is 
quoted from Bugelski (1938) describing the procedure previously. de- 
veloped by Skinner for establishing a bar-pressing habit more rapidly 
than would be possible ona strictly trial-and-error b 


that a response is more 
strength”) if, during extinction, 


ling) section, we have considered 


Here response- 
iven secondary- 


“preconditioned.” i.c., g 
to their "natural" occurrence in con- 
to be described is 


relevant stimuli were, so tO says 
reinforcement potency prior 
junction with the habit-to-be. 
a logical corollary of that pre 
(1956) as follows: 


The procedure now 


cedure, and is reported by Bugelski 


White (1953). designed an experiment 
Skinner box. He trained sev- 
previously described so that they were 
rats were trained they were given differ- 
ere placed in the box without a bar 


One of the writer's students, R- gra ii 
to discover the possible role of the click in the 
cral groups of rats in th 
efficient bar-pressers. When all the 


ent pre-extinction training. Some rats W ELA ME 
present and were presented with a click at an av erage interval of 30 seconds. 


The click was followed ordinarily by an approach to the food cup, now 
empty. As the clicks continued the fruitless trips ig. pw decline until they 
Stopped. altogether (usually within a period of gU che A: beau appear 
that. the food-cup approach was extinguished as far as the click was con- 
cerned. The next day the rats were replaced in the box with the bar present 
and underwent ordinary extinction (that is, no food was dropped to: them 
after the bar-presses). “Allowing for "spontaneous recovery and for the 
probable short-cireuiting of the behavior so that the click might not be 


c manner 
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particularly important if food were present, we could expect the rats to 
make some food-cup approaches, which they did. However, an interesting 
chain of events began to unfold. The rats began to cut down the number of 


trips to the food cup after bar-pressing, compared with a normal (non pre- 
extinguished ) group. Their behavior toward the food cup became less regu- 
lar, frequently they failed to reach it at all, and frequently they failed to 
“search” for food |see Fig. 7-7]. As the food-cup behavior began to break 
down, the entire pattern of return to the bar also weakened and broke down. 
chee dhe pre-extinction rats met an extinction criteria sooner than did the 
control animals (pp. 91-92). 


360 
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zo) Y sm 
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Fig. 7-7. Pxtinction of a bar-pressing response (and trips to food trough) in rats 
as a function of whether the secondary-reinforcing value of an associated click” 
had (Experimental Group) or had not (Control Group) been independently ex- 
tinguished. It appears that the secondary reinforcement provided by the “click” 
was an important component of the bar-pressing “habit.” The graph is reproduced 
from Bugelski (1956. p. 92). 


Bueelski himself interprets this finding according to an ultra-objec- 
tive theoretical model which has been discussed elsewhere (Mowrer 
& Keehn, 1958) and will not be further considered here (cf. Chapter 
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12, Section X). What is more immediately important is the way in 
Which such an outcome coincides with expectations generated by the 
of habit. Here we see that the strength of a habit can 
as this view would predict, without either non- 
ences thereof: all that is necessary is to 
(by extinction or countercondition- 


g to arouse secondary reinforcement 


—and the “habit” vanishes forthwith. This is hardly to be expected on 
at habit is a drive-behavior neural bond. This type of 
ant implications for the general subject 
wrer, 1960, Chapter 2) and 
a secondary reinforcement 


present view 
be lessened, precisely 
rewarded or punished occurri 
rob the response-correlated stimuli 

ing; sec Chapter 11) of their power 


the assumption th 
finding has, of course, import 
of “latent learning" (and extinction) (Mo 
0 ationship betweer 
). For immediate purposes, however, its 
1 needs no further elaboration. 


for the question of the rel 
and discrimination (Chapter 12 
bearing is sufficiently clear and 


VII. Supporting Evidence: 
(c) The Loucks Experiment 

bcen made to the "trick" of teaching a dog 
ly flexing the right foreleg and then ad- 
) some other primary or secondary re- 
ary reinforcement is presumably at- 
| leg flexion, with the result that 
making this response, because 


Reference has already 
to "shake hands" by passive 
ministering food, approval, € 
ward. By this procedure, secondary 
tached ro the stimuli associated witl 
presently the dog begins “voluntarily 


or 


nay remind the reader of the “sensory deprivation” 
o in Chapter 6 (Section HI) in which young men 
al visual, auditory, and tactile. stimulation 
with resulting "acute 


sep 3 ' . 

The foregoing discussion T 
experiments already alluded t 
arily deprived of norm 
1 , & Scott, 1956: Heron, 195 


were tempor: 


(Lilly, 1956; Heron, Doane X p nb : » 
disturbance of the normal personality ... swings in motivation... periods of 
a normal environment . . . handsome visual 


apathy ... desire to get back to a. i on 
hallucinations, disturbances of perception of the self, imp: r t of i j 
(Hebb, 1958. pp 110-111). Whatever else this type of "isolation involved, it 


certainly. constituted an interference with normal sensory feedback from the sub- 
ject’s ordinary behavior. Here, with the exception of proprioception, most of the 
sensory feedback was, in fact, eliminated, and i pur presepc anan 
it is not surprising (especially since the condition was maintained for many 


hours, or even days) that there were € . Ju 
would be like if all sensory feedback from action were eliminated, so that one 


could not tell if he had or had not performed a given action, whether he was 
in one bodily position or another, and so on! F rightening, sn't it? As the experi- 
ments cited suggest, we are here apparently dealing with variables which not only 
control specific “habits” but also one's very sense of identity and self-awareness. 
Clhis matter is further discussed, from a still different standpoint, in Mowrer, 


1960, Chapter 7). 


irment of intelligence" 


lysis is correct, 


disorganizing effects. Just imagine what it 
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of the secondary, or autistic (Mowrer, 1960, Chapter 3), satisfaction 
mediated by the stimulation produced by the occurrence thereof. 

Now the leg of a dog can be caused to flex and thus produce a char- 
acteristic pattern of stimuli by various means other than the one just 
described. One of these (see also Chapter 8) involves so-called faradic 
(electrical) stimulation of the dog's brain in the area of the cortex 
that controls this movement. In an article published by Loucks, in 
1935, and entitled, “The E perimental Delimitation of Neural Struc- 
tures Essential for Learning: The Attempt to Condition Striped Muscle 
Responses with Faradization of the Sigmoid Gyri,” the author tried to 
“condition” the response thus elicited to a buzzer which immediately 
preceded the faradic stimulation to the brain, with highly instructive 
results. Loucks begins his report by pointing out that: 


In a typical, striped muscle conditional reaction expe 
as a light is flashed for a second or so and then the 
eg. a faradic shock at | sic | the skin of the leg, is 
in a prompt movement of the limb. After 
signal light and cutaneous shock in combin 
shock evokes a movement of the leg. 
light in the absence of, or precedin 
response. 

The question arises as to whether or not all the 
involved in such modifications of behavior are 


riment, a signal such 
unconditional stimulus, 
administered. This results 
à number of repetitions of the 
ation, the light unaccompanied by 
This lifting of the leg at the flash of the 
g the shock, is designated a condtional 


various neural elements 
indispensable (pp. 5-6). 


Before we proceed with the further analysis of this investigation, 
two comments are in order. It will be noted, first of all, the Loucks 
speaks of conditional, rather than of conditioned, reactions. 
usage which has a good deal to recommend it (Mow rer & | 
1951 ); but it is not the common one and 
use in this volume. However, 


This is a 
AMOrCIUN, 
is not adopted for gencral 
no misunderstanding will result if the 
reader simply keeps in mind that the two w ords, as far as their techni- 
cal denotations go, are strictly equivalent. 

The other point to be noted is that Loucks, follow 
generally accepted at the time of w riting, 
as a conditioned, or “conditional,” n 
that, in the example 


ing the practice 
here speaks of leg flexion 
response. We would now 
given, the emotion of fear 
the response that had been conditioned 


a “voluntary” or "habitual" act. 


assume 
Was, strictly speaking, 
and that flexion of the leg was 
motivated by the fear. v 
again no difficulty need arise if one remembers that in 
leg flexion, when it occurs to the flash of light, as 
“conditional” ) response, Loucks was mere 3 
simplified way of speaking, 


However, 
referring to 
a conditioned (or 
ly conforming to an over- 
common at the time of u riting. 
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1 In a section headed, "Conditioning with Faradization of the Sigmoid 
Gyri as the Unconditional Stimulus,” Loucks then goes on to say: 


In the first of the experiments to be reported at this time, the objective was 


the stimulation of the motor cortex to sec if the limb response thus elicited 
could be conditioned to an auditory signal such as a buzzer. . . . This corti- 
cal faradization may be regarded as the unconditional stimulus inasmuch as 
It produces an involuntary and reflex-like movement of the limb. It is anal- 
ogous, in a sense, to the cutaneous shock at a limb in the usual experimental 
procedure. An obvious difference between this and the customary routine 
is that in directly. stimulating cerebral motor elements the investigator, in 
effect, short-circuits the afferent limb of the reflex. pathways that would 
normally be activated by cutaneous shock—disregarding for the moment 
secondary proprioceptive stimuli from the movement of the limb. This par- 
ticular phase of the project was taken as a starting point because of its techni- 
cal simplicity and because it was hoped that the role of the secondary pro- 
prioceptive stimuli, from movement of the limb, would be clarified (p. 7). 
Here the author is saying that in eliciting leg flexion by direct 
stimulation of the brain, the sensation of pain which is experienced by 
leg is prompted by peripheral shock 

but that “secondary proprioceptive 
will, on other hand, be nor- 


the dog when movement of the 
1a 7 A effet, short-circuited,” 
stimuli from the movement of the limb 
Mally experienced. 
Loucks continues: 

ation of unanesthetized and intact animals 
1 another place. - + - Briefly, it involves 
1 coil beneath the skin [of the head] and 
point to be stimulated [in the 
c the integument [body surface |. Subsequently, 
»v induction from the activated primary 


The experimenter can thus produce an 
lectrodes by pressing a button which 


external to the animal (pp. 7-8). 


The method of chronic faradiz 
has been described by the writer ir 
the embedding of a collodion-coatec 
n s from the coil to the 


ding insulated w 
ain}. No wires penet 
this buried coil will absorb energy t 
field" coil external to the animal. 
excitation at the bare tips of the buried elec 
causes current to flow through the field coil 
attempt to cause the buzzer to 


aration of buzzer and brain 


he results of his 


Loucks describes t 
igh paired preser 


clicit leg. flexion. throt 
stimulation thus: 

c what structures are essential for learning, an 
a movement of the right hind limb to an 


auditory signal, with faradization of the sigmoid pee ee the unconditional 
stint Ar conditional reaction of the hind limb failed to be established in 


any of three dogs receiving approximately 600 trials each (p. 42). 


In a project to determin 
attempt was made to condition 


a more dramatic substantiation of the view 


r involves fear arousal and relief, rather 
a formerly neutral stimulus with 


There could hardly be 
that active avoidance behave 


thar just the temporal conjunction of 
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the occurrence of some reflex reaction to a second stimulus. Here, in 
the Loucks experiment, the latter condition is fully met, vet there is 
not the slightest indication of the development of avoidance behavior; 
and the reason presumably is that the stimulus for the leg reflex is put 
into the nervous system in such a way that it can cause no pain. Without 
pain there is no possibility that fear will get attached to the signal; and 
without such a reaction to the signal, there is nothing to motivate and 
reinforce the occurrence of the specific leg movement (or any other 
overt act). As already noted in Chapter 4 (Section 1), Solomon & 
Wynne (1954; see also 1950) have shown that sympathectomizing an 
animal seriously impairs its capacity to react with fear and that an 
animal so treated shows a correspondingly impaired ca pacity for 
developing avoidance behavior. And in the same context, several other 


investigators are cited who have shown that sedatives and tranquilizers 
also depress avoidance behavior, 


But the Loucks experiment made the 
point, back in 1935, most convincingly of all: a reflex response made to 
a stimulus which cannot arouse pain (and fear) is utterly incapable of 
occurring anticipatorily, to a signal. A signal or “conditioned stimulu 
to be effective, must, it seems, be a danger signal. And if there is nothing 
about the unconditioned stimulus, avoidance 

behavior simply does not develop, even after * 
Woodbury (1942 


reports failure 


noxious or “dangerous” 


approximately 600 trig 
), in "A note on ‘passive’ conditioning,” likewise 
of “conditioning” in a situation wherein dogs were given 
a signal and then one of their legs was gently lifted by mechanic 
means, This writer refers to a 1928 experiment by Miller & Konorski 
in which it was found that a signal which was ‘followed by passive 
lifting of a dog's leg did acquire the capacity to clicit this ‘response. 
Woodbury repeated their procedure, with minor modifications, and 
failed to get conditioning, "In the course of 350 trials . . . there was 
no sign of such a change, either in the form of active flexion in response 


to the buzzer, or in a decrease in pressure on the treadle 
for the active lifting” (p. 360) 


in preparation 
S s . Woodbury suggests that Miller & 
Konorski may have gotten anticipatory: leg flexion in their experiment 
because they. vanked the leg up, roughly, by means of “a cord attached 
directly to the ankle, instead of lifting the surface on which the foot 
rested.” The yank on the string may ‘therefore have caused pain; and 
subsequent leg movement in response to the signal ms fee ‘been 
simply an ordinary avoidance reaction, which eld not have dc- 
veloped if the leg had been lifted more gently, 

Thus we have, in the Woodbury. ex 1 


experiment, a confirmation (in 
normal, unoperated 


dogs) of the Loucks finding that leg flexion which 
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which is in no way painful does not occur 
"avoidantly"—and why should it? If, on the other hand, a response 
were to be elicted in this manner (or by the method employed by 
Woodbury) and followed by a positive reinforcer (such as food), one 
might anticipate a quite different outcome (Section II). We now turn 


to the second part of Loucks’ investigation, which he reports as follows: 


were given biscuits at irregular 


is produced by a stimulus 


, The three dogs in the preceding group : 
intervals of four to ten trials | presumably as a means of making them content 
to stay in the conditioning stand, without struggling]. The routine for the 
two remaining animals differs from that described above in that food was 
given at the end of every trial. In other words, the buzzer was sounded for 
approximately. one second, the shock being added during the last tenth of 
the second, and then after a second or so, the animal would receive one or 


two biscuits (p. 16). 


Fig. 7-8. Schematization of the pro- 


cedure used in Louc econd cx- 


periment: The buzzer was presented Buzzer 
for one second, during the last onc- —— Faradization — COEM 
tenth = s s adic 

of which there was farac Leg flexion Ay 


ood " 
Food X 


stimulation of the sigmoid gyri. 
Was present (X) about one second 
After these two other forms of stim- 
ulation and leg flexion. 


The procedure just described is shown diagrammatically in Fig. 
Loucks savs of this part of his investi- 


7-8. In summarizing his findings. 
gaton; 


Lifting the hind limb to cortical shock was an wens bees for con- 
ditioning if each movement was rewarded with food. This was shown to be 
p ssociation, since once the response to 


dased upon : chani f backward a 
n a mechanism of bac k aci gs Ee 
the buzzer was extinguished, the animal failed to re-establish it if the food 


Was omitted (p. 42). 


ng theory, a different way of 


yr learni a 
itself. This investigator infers 


, In light of revised two-factc 
interpreting Loucks’ findings suggests 


that the subject “formed the condition T , 
a backward association with the food rew ard." Just how this would 
(see Chapter 10). But a different type of 


ntal facts readily comprehensible. Ac- 
Id see in this experiment, not an 


al response upon the basis of 


occur is not entirely clear 
explanation makes the experime 
Cording to present theory, we wou : A 1 

£ ard conditioning at all, but rather a straightforw ard 
Loucks speaks of the “centripital volleys 


the higher somesthetic nuclei” which 


Instance of backw 
Case of forward conditioning. 
ot proprioceptive impulses to 
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were excited by each flexion of the leg by means of faradic stimulation. 
And each of these “volleys of proprioceptive impulses” was follow ed 
by food, cating, and hunger reduction. Therefore, the proprioceptive 
stimuli produced by the leg flexion caused by cortical stimulation 
would take on secondary reinforcement, which would then gen- 
eralize, probably quite strongly, to the proprioceptive stimuli pro- 
duced bv normal, "voluntary" leg flexion (see Fig. 7-9). In short, 
the procedure described by Loucks is analogous to that used to teach 
a hungry dog to "shake hands" with one of its forepaws and, in light 


of our earlier discussion, would be beautifully calculated. to establish 
à “habits” 


Fig. 7-9. A more complete 
analvsis of the procedure de- 
picted in Fig. 7-8. Here is 
shown, by means of the 
spike, the leg flexion pro- 
duced by the faradic stimu- 
lation. “This response pro- 


Buzzer L duces certain response-cor- 
Faradization n related stimuli, S, Also 
Leg flexion X Sre shown are the reduction in 


hunger and the reduction in 
hunger fear produced by 


Hunger et the food. The crucial learn- 


ing here consists, presuma- 
Hunger bly, of the reduction in 
Ns hunger fear (or secondary 
reinforcement, | decremental 
type) becoming conditioned 
to the response-correlated 
stimuli, S}. Per hypothesis, 
these conditions meet the 
requirements for the estab- 
lishment of a leg-flexing 
“habit.” 


Food X 


There is onlv one complication in the foregoing anı 
procedure employ [d 


by hunger and 


ysis: If the 
d by Loucks produced a regular “habit. motivated 
reinforced by hunger reduction, 


ee quite specifically, in response to the buzzer? Our surmise 
would be that in this situation the buzzer really functioned more as 


a cue than as a signal. During training the dog would have found that 


voluntary: flexion of the leg would produce food only if the buzzer 
had just sounded; so the dog would ace ' 


why did it occur, 


ordingly form a discrimina- 
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ation-without-buzzer and experimental- 


tion between experimental-situ 
the former being “negative” 


situation-w hen-buzzer-has-just-sounded, 
and the latter being "positive" as far as response effectiveness was 


concerned. 

If this interpretation is correct, 
process of forming such a discrimir 
number of wrong or false responses. 


right hind leg between trials, when the buzzer 
soucks makes no reference in the text of his paper to such interven- 
` Jee flexions; but the legend of one kymographic 


one would expect the dog, in the 
ation, to have made a goodly 
in the sense of often flexing the 
cue was ot present. 


ing or “spontancous’ 
record reads: 

fed crackers between. cach trial 
d leg flexion]. Definite condi- 


„First phase of training where dog was 
ic, immediately after each cortically elicite ; n]. De co 
tional movement in right hind leg at cach buzzer signal. Conditional lifting a 
Sustained movement, “whereas unconditional response to cortical shock a 
relatively brief swing of short latency. A further difference between the 
conditional and unconditional movements is that the letter was obviously a 


ifting of the limb, the former a swing or kick (p. 12). 


c "conditional" leg flexions 
came on, but also relatively frequent 
then, is direct support for the 
made to the buzzer was ac- 


And in this record onc not only sees th 


Which occurred when the buzzer 
Interval, or "spontaneous," flexions. Here. 


assumption that the response ostensibly f s ac 
tually motivated by hunger and chat the Duzzer was jase acne. indi- 
cating when the response would and would not be res arded. One can 
i ble confidence that as training proceeded, the 
onses diminished 
(right) responses 


predict with considera 
Interval (wrong) resp 
While the “conditional” 
pter 12). i 

aditional habit the 


in frequency of occurrence 
increased in frequency— 


and specificity (cf. Cha 

It is not clear how tr 
reported by Loucks; and i 
tional conditioning theory. 
phenomenon. of backward 
to order" 


orv would explain the results 
n order to interpret them in terms of tradi- 
one has to posit the highly questionable 
conditioning. But the findings, as just 
shown, are “made as far as revised two-factor theory is 
Concerned. : 

In a third and final part of thi 
interest to us here. Loucks found th 
in addition to eliciting 2 movement o 
a conscious sensation of some kind, 


fornio simulation could be made to act as : s 1 i 
UP onam dog ee a result of its being paired with ordi- 
> of the dog? 


" > There was, of course, a possibility 
nary cutancous shock r ] 


s study, which is of only incidental 
at faradization of the sigmoid gyri, 
f the dog's hind leg. also produced 
as indicated by the fact that this 


tas a signal, or CS, for flexion 


nt legs. as 
to that leg. 
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that it was not the direct perception of brain stimulation as such that 
was here acting as the conditioned stimulus but instead the proprio- 
ceptive “back-lash” from the resulting flexion of the hind leg. In order 
to check this possibility. Loucks simply deafferentated the hind leg, 
so that the dog had no sensations of touch or movement therefrom; 
but still the brain stimulation acted as a CS for foreleg flexion, under 
threat of ordinary electric shock to this leg (cf. Nielson, Doty, & 
Rutledge, 1958). 

The over-all import of Loucks ingenious researches is therefore to 
show the necessity of secondary reinforcement, in one for 
for the development of a “habitual” response. In the e 
described, secondary reinforcement type-1 w 
currence of foreleg flexion by reduction in 


m or another, 
xperiment just 
às provided for the oc- 


the fear aroused by the 
"buz" produced bv sigmoid gyri stimulation; this, therefore, was 
simply an instance of ordinary active avoidance learning. And, in thc 
earlier part of Loucks’ study, the dogs learned to lift the hind leg, on 
cue, because the proprioceptive, cutancous, and other stimuli pro- 
duced by this response had come to signalize secondary reinforcement 
type-2, i.e., hope of receiving food. The result was a 
criminative habit. 

An excerpt from Loucks’ 
ings follows: 


an ordinary dis- 
own discussion of his experimental find- 


One of the most striking facts arising from these experiments is the in- 
adequacy of mere repetition, as such, for learning. Although the 
given hundreds of trials in which an auditory stimulus wa: 
a limb response, the repetition of thes : 
inadequate to bring 


dogs were 
sociated with 
€ two events, in quick succession, was 
t about learning. This would seem to necessitate a rein- 
terpretation of the Law of Exercise as usually formulated in the psychology 
of learning. It indicates the serious inadequacy of those formulations of 
learning or conditioning built solely upon the principle of association by 
contiguity. It would appear that mere repetition, as such, is not enough, Con- 
Uguity is not enough (pp. 34735). k 


And, in conclusion, Loucks says: 


_ Various experiments in which conditioning has taken place when only 
limited portions of the normal, reflex pathw a : 
view ed to show. that conditioning is established upon the basis of certain 
primitive urges, impulses, appetites—probably subcortical in character—and 
that it is the feeling of pain, hunger, nausea, etea, which is the significant 
factor rather than the reflex response which is merely one com TEAN ina 
complex pattern (p. 42). É 


ys were functioning were re- 


It would thus appear th 


at Loucks’ em 
congruent with expect 


i pirical findings are completely 
ations generated 


by revised two-factor theory 
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and that this theory is a logical culmination of interpretative trends 


to be noted in the passages last quoted. 


VIII. Supporting Evidence: 
(d) Delayed Auditory Feedback and 
Related Phenomena 


Writing in The Scientific Monthly, for February, 1953, Stanton 
and Sylva Cohn, drew the following cogent distinction: 


“Age of Power.” It saw the development 


The nineteenth century was the 
ith it there arose a mechanistic philos- 


of the machine, and concomitant wW 
ophy of life and a mechanical interpretation of life processes. . . . 
Science has advanced beyond the mechanistic stage, however. Just as the 
Nineteenth century was the Age of Power, the twentieth century is the Age 
of Communication and Control. It is not enough to make a powerful machine, 
having the ability to do many times the work of man. There must be an in- 


telligent application of this energy—it must be controlled (p. 87). 


Nineteenth-centurv. power machines were typically what are now 
being called open-cycle control systems: a button was pushed or a 
lever thrown here and an effect (considerably magnified in terms of 
energy) appeared there And this was the model of stimulus-and- 
response functioning which tacitly underlay the classical conceptions 
Of the reflex and of habit, as they have existed in physiology and in 
Psychology. By contrast, the twentieth century has produced, in in- 
creasing ‘profusion, machines which operate as closed-cycle systems, 
ic. Which are self-regulatory (cf. Mowrer, 1960, Chapter 7). Mani- 
festly, the latter provide a much more realistic model for the behavior 
which are so exquisitely self-regulatory and 
than did carlier mechanical conceptions. 
ry, ds a conception of “habit” 
per theory, as intelligent as 


uF living organisms, 
‘homeostatic (Chapter 6), 
mid-twentieth centu 
anisms, 
for that matter, as intelligent as are 
himself has recently perfected. 

aper by the present w riter which re- 


But only now, at 
volving which makes living org 
they manifestly are in action gd, 
a man 


ofap 


ats, with special reference to the dramatic 


some of the machines whicl 

Ihe following is a portion 
Views some of these developmet D 
effects produced by delaved auditory feedback. 

What this line of evidence seems to add up to. then, is the inference that 

sponses become “habitual” by virtue of the proprioceptive and other stimu- 
lation associated therewith having acquired secondary reinforcement value. 
This formulation, paradoxical as it sounds, seems to apply with equal cogency 
regardless of whether we are speaking of a “habit” (a) which is initiated by 
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the learner, as in trial-and-error behavior, (b) one in which the subject is 
"put through" a given performance, pas ely, or (c) one which is first exe- 
cuted by azotber organism with resulting stimulation which the subject, on 
the basis of secondary reinforcement, subsequently tries to reproduce | see 
Mowrer, 1960, Chapter 3 |. A conception which thus accounts, economically 
yet adequately, for the learning involved in trial and error, in Į t 
through," and in "imitation" has, it seems, some claim to serious consideration. 

If we assume that a "habit" or "habit strength" depends, not upon the de- 
grec of neural connectedness between a given drive and a drive-reducing 
response, but upon the amount of secondary reinforcement that has been 
built up by association of incidental stimuli with other reinforcing states of 
affairs, it follows that a "habit" will be more seriously disturbed by inter- 
ference with the sensory “feed-back” therefrom than by interference with 
specific stimulus-response bonds. Certainly brain ablation experiments have 
not given results which look much like what would be expected if habits 
were just acquired reflexes. Especially pointed are the results obtained by 
means of various forms of peripheral interference, In an carly series of exper 
ments, Lashley & Mc rthy (1926) showed that one can radically impair 
the locomotor facilities of a rat without impairing a maze “habit.” And it is 
also known that if, on the other hand, the sensory return from the bodily 
members involved in behavior is climinated, habitual performances are s 
ously disorganized. For example, Light & Gantt (1936) have shown that a 
leg-flexing, paw-lifting habit can be taught to a dog while the limb in ques- 
tion is completely: ralvzed. Under these conditions, what the dog learns 
is that a given signal is likely to be followed by an electric charge upon the 
grill on which the paw is resting; and when voluntary control of the limb 
recovers and the signal is presented, the animal makes an adaptive flexion 


response—a response which will now occur “habitually” despite the f. 
that occurrence of the 


so-called habit,“ 


assive "putting 


act 
response was not involved in the acquisition of the 


And it is well known that locomotor habits in human beings 


T Superficially, these observations appear, in one respect, to go against the 
present view of habit. To be sure, they indicate that “performance A 
to this kind of learning; but they also seem to show that neith 
feedback from behavior (ef. Beck & Doty. 1957). However, this misleading im- 
Pression is easily corrected. In. the Lashley MeCarthy experiment, although the 
subjects’ mode of locomotion—and hence the kinesthetic and tactile feedback 
therefrom —was radically disturbed without serious impairment of the maze 

habit,” ir must be remembered that the visual. 
associated forms of Simulation Were not affected and that the "habit" could 
easily have been carried, or retained, by these 
esthetic and tactile. stimuli, Response-correl 
here conceived, are by no 
what of the Light-Gantr 
could be establish 


is not essential 


er is the sensory 


olfactory, and possibly other 


» rather than necessarily by the kin- 
ated (response-producd) stimuli, as 
means restricted to stimuli of the latter kinds. But 
study? H it was found that an 
ed without occurrence (or even * 
The issues here involved are clarified by an 
t1930). This investigator found that c 
established with morphine as the 


avoidance habit 
‘putting through") of response. 
earlier study reported əy Crisler 
a conditioned salivary 
UnCS even though, 
om occurring by. 


response could bc 
during conditioning, the 


salivary response was prevented fri the use of atropine (for a 
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are spectacularly disorganized in tabes dorsalis—a condition in which the 
motor nerves to the legs are fully intact but one in which the proprioceptive 


return therefrom is lost. 
here is, however, a different, and even neater, type of observation. If a 


ked to speak, he can do so with little or no distortion 
g dback is cut off. This feat is accomplished, pre- 
sumably, on the basis of bone-conducted sound and proprioceptive stimu- 
lation from the throat, tongue, lips and jaw. But now let us re-introduce the 
alr-borne feedback, on w hich the subject ordinarily places greatest reliance 


In Monitoring his speech, and see ction 
(egs 14) of a second, as can be done by "storir 
On a magnetic tape before allowing it to get back to the speaker. 
by Lee (1950), Black. (1951). Fairbanks (1954, 1933). and F 
Jaeger (1951), and others reveal a ver) dramatic result: Despite its highly 
practiced, “habitual” nature, under these circumstances human speech is 
immediately and almost invariably distorted, often. quite ludicrously, and 
Sometimes becomes so disorganized as to be utterly. chaotic! * 

On the basis of the traditional SCR interpretation of habit, this remarkable 
effect has been quite incomprehensible. Can it be that in our re-examination 
of habit theory we have stumbled upon its true explanation? This effect is 
Very possibly ‘a better test of our hypothesis than would be any effort to 
exclude all sensory. feedback from a habitual response system by means of 


normal human being is a 
even though all air-borne fee 


what happens if we delay it by a fra 
ng” the speech momentarily 
xperiments 
irbanks and 


review of other related studies, sec Morgan & Stellar. 1950. pp- 453 E). However, 
inst the present over-all conception. of learning, this 
assuming that bait involves the condition- 


Conditioning itself is jor 


far from being evidence 
finding neatly supports it. W 
Ing of hope to certain response- 
assumed ro operare this way: it is ¢ 


diti y $ : 5 
litioned and unconditioned stimulus (cf. Ch 
the Light-Gantt exp 


undoubtedly alres 
anger of being 


here 
correlated stimuli. 
lependent solely upon the pairing of the con- 
apter tt, Section V). Therefore, what 
»eriment was this. The dog. as a 
learned what to do when its 


re i 
Presumably: happened in 
res [5 

sult of past experience, had 


as in d 
‘Thus, one may say 


was that an already. existent 
jure but could not be manifested until the motor nerve 

therefore—in fact, virtually certain—that no 
t to a response which had had no prior 
here success could very likely be 
arousing 


ady 
hurt) by the surface on which it 
that all that happened in the 
“habit? was energized by 


ees being hurt (or wa 
[ion CAMS poe dO retract it. 
ms inm experiment 
| pecifie training procedur 
ad regenerated. Ir seems like 
such effect would be obtained with respec 
Practice, or "habit strength.” However even hi 
attained by (a) pairing ^a signal with shock (in order to give it fear- S 
Potency) and (b), while the motor nerve was still severed, using the putting- 
i 5 sociating cermin response-correlated stimuli with signal 
Upon recovery of the motor nerve, such a "habit" 


rance de novo! 


th ^ 
: rough technique for as T 
*rmination and fear reduction. 


nmi $ 
ight very well make its app j 
* A qo-inch L-P recording of this y, with explanatory. comment, may 


be obtained free af charge (through courtesy of John Wiley & Sons) by writing 
to the author of this book. An extensive bibliography (Sutton, Chase, & First, 


los ^ c 2 3 3 
959) cin De.obtaied-from Di: R à Chase, : 
are also. studying delayed feedback in other sensory 


phenomenor 


Biometrics Institute, New York City. 


D H i 
t; Chase and associntes 
modalities, 
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anaesthesia, nerve 


tioning, or the like. In this procedure, with the subject 
in an otherwise completely normal state, we can mask out, apparently s 
effectively, all (or certainly most ) subsidiary sensory feedback by the trick 
of slightly delaving, or throwing "out of phase," the major sensory return 
from the activity in question. Anyone who has experienced the effect is 
impressed, not to ay appalled, by it, as pure phenomenon. And in the present 
context we are likewise struck by how satisfactorily this otherwise baffling re- 
sult is accounted for by what is to us an unaccustomed, but intriguing, con- 
ception of the mechanism of "habit." . 

In conversation Professor Fairbanks has reported that with the 
mentioned above, observations have been made 
instruments, rather than speaking: c.g. 
the only auditory feedback occurring via a microphone, the “dela 
mechanism, and a set of headphones which exclude all other (i.c 
auditorv stimulation. Under these conditions, the h 
forming on a musical instrument arc disorg: 


as are speech habits. Dr. Donald J. Mason has suggested another interesting 
possibility: namely that. of switching the out-put of an electronic orgran 
from its loudspeaker to the tape delaver and then connecting the speaker to 
the output of the delaver. This would give the desired 

would eliminate the necessity of having the 
thus making the situation more “natural.” 
back from the movements involved in pla 
and, to a lesser extent, visual. With the m 
out of phase, the organist would 


sanc set-up 
with persons pl ing musical 
«a violinist is asked to play but w ith 
ing 
«the direct) 
abits involved. in per- 
anized in much the same manner 


asynchronism and yet 
performer wear headphones, 
The only direct, immediate feed- 
ving the organ would be kinesthetic 
ajor feedback, i.e., the audito ry one, 
almost certainly find it impossible to play 
normally on the basis of proprioceptive and visual cues alone—particularly 
if the composition were a rapid one. Chimes players and public-events an- 
nouncers using a public-address system with one or more remote loudspeak 
ers also report "habit interference" because of the (acoustically: ) delayed 
feedback (Mow rer, 1953b, pp. 12-15; see also Chase, 1959.) i 


The facts reviewed in this 
by no means exhaust the c 
what, for want of a better term, has here 
factor conception of habit. And in | 
both a direct and indirect n 
also Mowrer, 1960). 
delineated 


and the preceding sections of this chapter 


vidence that can be adduced in support of 


been called the revised two- 
ater chapters further evidence, of 
ature, will in fact be extensively cited (see 
But the theory itself has now been: sufficiently 


and enough empirical justification for it presented to war- 
rant using the next two chapters to show its rel 


ceptual schemes. The success or failure of the 
important respect, will hinge 
unifies the view of others. 


ationship to other con- 
present theory, in one 
upon how well it articulates with and 


Upon reading a typescript copy of this chapter, Dr. D. W. Zimmerman made 
the following comment argument in this chapter are Rirchie's 


His argument, advanced from a somewhat 
ame import. 


“Relevant to the 
(igs4) criticisms of the S—R model. 
different standpoint, has much the s The revised conception of habit 
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IX. Summary and Comment 

ounded out,” in at least one phase of its 
development, the extensive program of research and logical analysis 
Which has been traced in the preceding chapters. As it gradually ‘be- 
r of the two great behavioristic conceptions 
ioning and habit formation in the 
al interest developed in 


In this chapter we have Y 


came apparent that neithe 
of learning (classical Pavlovian conditi 
manner of Thorndike) was satisfactory, speci 
active avoidance learning, with the resulting discovery that fear (which 
had previously been barred fr ation as too subjective and 
Unscientific) must be posited 
New conception of passive av 


rom consider 
as an intervening variable. This done, a 


'oidance learning (punishment) emerged, 


whic 1 i i 
^ hich quickly superseded (or. rather, integrated) the views of both 
avlov and Thorndike in this arca. And it was at this stage that the 


econd version of two-factor learning theory came into currency. 
iginal formulations of Pavlov and 


But while an advance over the or! 
constituted the first version of 


Thorndike (which, in conjunction, 
this schematization also had its troubles, among 
attention to “positive” approach be- 


; a have been investigated (especially 
in the past i0 or 15 vct c to distinguish systematically 
between secondary reinforcement. 1y pe! (as it occurs in active and 
passive avoidance learning) and secondary reinforcement ty pe-2 (as it 
occurs in approach behavior). And at this point we discovered that a 
New and vastly improved conception of habit had suddenly dropped 


p 


o met theory), 
the one ; : 
aem the fact that it paid so little 


havior f 
lavior, As this and related phenomen 
;), we have com 


also leads to the necessity of re-examining 


argue, 7 
ation of 


chological theory. 
adequate as a model for the kind of 


however, is more congruous 


deve 1 š i 
x cloped in this book, one might 
Ne logic psy 
logic and methodology of psy 
R = f(S), is no longer 


1 here. Ritch position, 
Be needed" The present author is not sufficiently 


c's argument to endorse Zimmerman's evaluation, 
but it is reproduced here as suggestive. This much, however, is certain: The view 

a response must bc replaced with the notion that what 
can, in most instances, be better termed 


Spence's classic descr 


Dyechological laws, as I 
origi dynamics develope 
) a hat would seem to 
Certain that he understands Ritchi 


tha F T 
At og stimulus “causes 
alled a "response 
a great many 


ponse” or 


S—R sequences in which "S" is pro- 
duced by the just preceding ` segment of the total act. The reader 

this for himself as follows. Take a straight-edge and 
Then, starting at A. quickly trace A-B freehand. 
A-B and returns thereto which the tracing 
of tracing A-B consists, actually, of a 


we i 
have previously € 
an ae, i is co 

Tet, Which is composed of 


Can easily 

i easily demonstrate all 
raw ali r 

p aw a line, A-B, one foot long. 
Pr ; r 1 
om the many slight deviations from 


wi P E: 2 
ill show, it will be evident that the total act 


For further discussion of this. problem, see Mowrer 


succe. 3 ^ 
1 ecession of "responses. 
1960, Chapter 7). 
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into our hands. It is the latter, of course, which has been elaborated in 
the present chapter, with considerable array of both old and new lines 
of supporting evidence. j 

However, despite the logical fit of this conception of habit and its 
widespread empirical justification, it has, thus far, received little formal 
recognition. For example, it is not even mentioned in Kendler’s review 
of “Learning” in the 1959 Annual review of psychology—although a 
great deal of research is therein described which fairly cries for this 
type of interpretation, and new findings are constantly appearing in the 
literature which quite specifically support it. Kendler refers to “the 
stark inadequacies and limitations of existing learning theories" (p. 43) 
but makes no use of what would now appear to be the most powerfully 
integrative system of all. His concluding remarks are nevertheless in- 
structive: 


Learning psychologists are still groping with methodological problems 
(both in the philosophy of science as well as in the experimental sense) that 
they hope will lead to a major breakthrough in understanding behavior. 
Some believe a breakthrough is now occurring. But nobody is 
recognize anybody's breakthrough but his own. i 
functional and comprehensive syithesi | 

However, the gropings of the past vear do suggest a pattern. The major 
features of this pattern will now be given ina modestly pontifical fashion. 

(a) The range of empirical phenomena that learning theorists are now 
trying to handle is becoming wider and wider with motivation and per- 
ception (at least orienting acts) playing a larger and larger role. Truly, 
EN theories are behavior theories. |Sce Chapters 8 and 9; also Mowrer, 
1960. 


(b) S—R psychology. dominates the psychology of learning. The success 


of S—R psychology is partly due to the fact that it forces its user to think in 


terms of manipulable experimental variables and observable responses. | Cf. 
Chapter 5, Section IX.] 


(c) Yesterday's physiologic 
to pay off in generating rese: 


is willing to 
| Cf. our efforts to achieve a 


al models of the learning process have failed 
ia arch. Physiology, however, is not becoming less 
m epu m ut rather more Important to the psychology of learning. The 
reat advance at are 3 relat d Vn d D 
grar * vances that are now taking place in experimental psychology are, 
and will continue, establishing a true rapprochement between learning theory 
and physiology. | Cf. Chapter 6.| i 

(d) Learning theories are paying more and more attention to the Ss 
response. The dream that be 


; havior could be simply related to such in- 
dependent variables as delay of reinforcement | Chap 


— Chapter 5| has been Shattered. It has been demonstrated (Spence, 
Pe $y t ma PARI delay which is the important variable but what the 
n es during the delay. Se : i i 
dy during delav. The consummatory response is probably more im- 
portant than the amount consumed (Wike & Barrientos 1957). Guthrie 


nd be given credit for em this point many years ago | Chap- 
ter 8|. Dc d 


apter ro] or amount of 


phasizing 
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techniques are becoming more widely 
measure seems to be at times 
an traditional measures of be- 


(e) Skinnerian experimental 
adopted | Chapter. 8]. The rate of response 
(Miller, 1957) a more sensitive measure th 
havior. i 

(f) Complicated experimental designs involving complex statistical pro- 
cedures seem to be offered at times in lieu of theoretical notions. 

(g) In general, learning theorists understand each other much better than 
did their ancestors of two decades ago. Neobehaviorists, SR functionalists, 
and statistical learning theorists can communicate ea ly with each other. 
Skinnerians also find it easy to communicate among themselves. 


(5) Progress is being ma le! (pp. 78-79)- 


‘hich Kendler finds in con- 
all the more remarkable 
we have found so useful 
after 


In view of the widespread confusion W 
cory and research, it is 
erence to the scheme 
o it). But perhaps there is 


temporary learning th 
that he makes so little ref 
(only in d is there tangential allusion t 
all, a reason! In advancing a feedback conception of both response in- 
hibition ( punishment) and response facilitation (habit), we have eman- 
Cipated behavior theory from what may be called the “bondage” of 
Thorndike’s scheme and also liberated it from the crass reflexology of 
. in one respect this “revolution” may appear to have 
gone too far. Once a response which has previously been rewarded or 
Punished is in progress (either actively or syftibolically--see Mowrer, 
1960, Chapters 3-6), we readily scc how it can be facilitated or in- 
hibited, respectively, by hope or fear. But if we thus put all learning 
Gie., the conditioning of hopes and fears) over on the side of the sens 
F seem to be left without any explanation 
ain responses rather than others do, 
irted in, lC apparently become "connected 
Why, in other words, are these particular 
N usands of other possible 


Dav 
Pavlov, However 


sory Consequences of an act, WC 
Whatever as to how it is that cert 
after all, manage to gef st 
With," a given situation. 

responses, rather than hundreds or even tho 


reactions, selected and initiated? 
© Vhorndikian connectionism h 
Ing this problem, since it assumed 
response bond, in the sense of a (neura 
drive or situation (when it occurs) and 


But now, for good and sufficient reasons, V ch dt 
don this theory; and, as We have seen, the one we have put in its stead 


has all the advantages that could be asked—except for the critical ques- 


i - I p NET 
ton of response selection and initiation? | M 
f an expositional dilemma. The writer believes 


At this point we face 1 l bl : 
“cist ion to the problem just posed; 
iitous and unexpected manner 


derable merit of avoid- 


ad the very consi 
ata habit is a stimulus- 


, quite simply. th 
Jz) connection between a given 


a particular response tendency. 
ave been forced to aban- 


actory solut 


here is an eminently 52 À 
3 ather circu 


but it has been arrived at in a T 
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and cannot be convincingly described at this juncture without a major 
digression. Therefore, what we shall do is deliberately postpone full 
discussion of this problem until later (Mowrer, 1960, see especially 
Chapters 6 and 7) and proceed in the remainder of this volume with 


the analysis of various other problems which more naturally articulate 
with what has already been s 


aid and which do not depend, in any 
critical manner, upon the resolution of this dilemma. 
therefore, will be to ask the reader to take it "on faith" that a solution 
to this basic problem is available, with only the bricfest intimation here 
of what it specifically involves. 

As we have now seen, the whole history of Beh 
in the direction of libera ization (cf. Miller, 
that Kendler, in his review, hardly refe 
to speak of “neobehaviorism” and 
movements readily accommod 
Which pristine Behaviorism wo 
intimated by Kendler, 
acknowledge 


The strategy, 


aviorism has been 
19593), so much so, in fact, 
rs to it at all, preferring instead 
“S-R functionalism.” These latter 
ate concepts, such as fear and hope, 
uld not countenance. Morcover, as also 
these movements are now even beginning to 
“perception (or at least orienting acts)." In other words, 
the relevance of cognitive as well as affective processes is being recog- 
nized in systematic theory; and the solution to the problem of response 
selection and initiation hinges, quite specifically it seems, upon the 
reality of imagery (or memory), which is a cognitive phenomenon, 
pure and simple. If, how ever, we now plunged into a full-scale con- 
sideration of cognitive (and symbolic) processes, we would by-pass 


topics of real import: to which we would then later find it difficult 
and somewhat artificial to return. Therefo 
plan will be to hold discussion for the b 
such that cognitive consider 
sequently, 

With th 
further dey 
term—may 


nce, 


e re, às already indicated, our 


alance of this volume at a level 
ations can be side-tr 
returned to for detailed t 
is explanation, we DOW proceed, 
‘clopment and defense of wh 
be called Two-Factor | 
simply, revised two-factor theory, 


acked and then, sub- 
reatment. 

in the next chapter, to 
at—for want of a more graphic 
«earning Theory, Model HI or, 


Other Theories, 
and Some Further Evidence, 
Compared 


to present a particular conception 
a manner as possible, This con- 
ews of other writers, 


Thus far, an effort has been made 
Of learning in as explicit and distinctive 
ception has been carefully contrasted with the vi 
^g ambiguity and of posing logical and em- 
Pirical issues with maximal clarity Now, however, the time has come 
to look for agreements and congruences. And since one of the avowed 


aims of this book is to delineate 2 position which "integrates" scem- 
ne, then naturally, the wore agreements 


in the SERA Re Get 
the hope of climinating 


ingly diverse theories of learnir 
that can be found the better! $ 
Sometimes it has been said to the author, "But isn't vour theory a 
lot like that of Dr. N or Prof. Y?" Two disclaimers are in order here. 
ition which is being set forth 


I he author very much hopes that the positi hic e f 
in this book will not be regarded as “his, since it is specifically aimed 


at finding the common ground toward which the thought and research 
ofar Ncarkars Nive for some time been moving. Moreover, in light 
3 act that there are similarities between the posi- 
t of other learning theorists is decidedly to 
the good. Prevailing disparity of views is not the hallmark of a great 
science; and it will now be our purpose to show the very substantial 
unity which has already been reached in this area and to suggest the 
even larger and more rigorous integrations that may be achieved in 
253 


Of this ` 3 s 
f this objective, the f 
tion here adduced and tha 
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" ever, it will be useful to make an explicit, sum- 
the future. First, eo how the present position differs from carlier 
mary napi acini P theory. 


versions of 


he Three Successive Versions of 
y T eae learning Theory 


Thorndike’s Law of Fffect, as formulated around the turn of the 
century, is schematized in Fig. 8-1; and Pavlov's conception of con- 
ditioning is represented in Fig. 8-2. Fach alone, according to its ex- 
ponents, was capable of accounting for the basic facts of learnings; 
but many others felt that both conceptions were neces v for a well- 
rounded account of behavior dynamics. Thus, what we may call the 
original two-factor learning theory assumed that conditioning and 
problem solving played equally useful, complementary roles 
total adjustmental economy. of living organisms and that c; 
was inadequate, conceptually and practically, without the 


in the 
ch alone 
other. 


84—— — — — —— R,:reward 84 — — — — — R: punishment 


S; — — R; S, 


Fig. 8-1. Schematic representation of the two halves, or 
Law of Effect(s). It held, in essence, th: 
instrumental (behavioral) response R,, 
SiR, sequence, or bond, will as 
that if such a stimulus-response 
will as a result be weakene. 


aspects, of “Vhorndike’s 
at if some drive stimulus S, produces some 
and if R, is then followed by reward, this 
à result be strengthened (diagram at left) 
sequence is followed by punishment 


d (diagram at right). 


and 
» the bond 


Si ———__—__=R Se— R 
Fig. 8-2. Schematic representation 


of conditioning, 
is a stimulus which, initially, 


is not capable of cliciti 
a result of contiguous occurrence with S, 


(above right). The subscripts c and u are 
Pavlov termed the "conditioned" and the 


as conceived by Pavlov. S, 
ng response R, but which as 
left). acquires this capacity 
used to designate, respectively, what 
“unconditioned” stimulus. 


(above, 


OTHER THEORIES AND EVIDENCE COMPARED 255 


The second version of two-factor theory (actually the first one to 
which this special nomenclature was applied ) accepted, essentially 
unmodified, the “first half” of the Law of Effect, which held that re- 
ward strengthens habit; but it departed therefrom in holding that 
punishment achieves its action, not simply by reversing the effects of 
reward, but by causing fears to become attached (conditioned) to 
stimulation associated with the occurrence of the punished response. 
This version of two-factor theory is shown in Fig. 8-3. Here habit 
arning, is conceived essentially as Thorndike 
suggested; but punishment is seen in more complex terms. Instead 
of reversing the effect produced by past reward, punishment is here 
thought of as providing the basis for the conditioning of fear to 
j which, when the response starts to recur, 


formation, or solution le 


response-correlated stimuli, 
and. if the fear is strong enough, response inhibition. 


ation of punishment are shown at the 
an 


produces conflict 

The details of this interpret à 
3. Here a problem situation or drive, Sa produc 
R, Ri is followed by punishment, which 
punishment clicits a response R, of which 
; occurs, it not only elicits the 


right in Fig. 8-3 
overt, instrumental response, 
is here represented as S,. The 
component. But when R i j i 
it also produces a number of other stimuli, 


associated with the occurrence of Ry. The result: 
roduced by Sp namely fear, gets conditioned 
1 stimuli, s, s, s. Consequently, when Sq re- 
to perform Ri, the resulting stimuli "re- 
antecedent punishment, ie, cue off ry, 
f. Chapter 3. Section V). 

d version, thus accepted Thorn- 


re or fear, is à 
extrinsic stimulus, Sp; 
which are inherently 
à part of thc reaction p 
to thesc responsce-producec 
curs and the organism starts 
mind" the organism of the 
which tends to inhibit Ry (€ 

Two-factor learning theory, secon 


S;——— R; :reward Sa 
d 


sı — Ri 


Fig. 8-3. Schematic representation of the second version of two-factor learning 
According to this 


in essentially the il 
avolve fe 


as shown at 


formulation, habits were made by reward, 


theory 
$ wav suggested by the Law of Effect However, pun- 


the lef 
ishment was assumed to i 
conflict and probable inhibition (see text). 


ar conditioning, as shown ar right, with ensuing 
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dike’s conception of habit formation (essentially unmodified) but 
derived the phenomenon of punishment from fear conditioning, rather 
than attributing it to a process which simply reversed the effects of past 
reward. As we have shown, this theory of punishment (or passive 
avoidance learning) involves the same principles that have been 
found most satisfactory for explaining. avoidance learning. (of the 
active variety ). In both instances, fear becomes conditioned to stimuli, 
either external or response-produced; and the organism then makes 
whatever type of adjustment will most effectively eliminate. these 
stimuli and reduce the attendant fear. 

The present-day version of two-factor theory is diagrammed in 
Fig. 8-4. Here it is assumed that habit formation is a matter of con- 
ditioning no less than is punishment. If a stimulus 
response Rj, and if Ry is followed by reward S, then it is assumed 
that a part of the total response R, which is produced by S, will be- 
come conditioned to the stimuli inherently connected with Rj. 
conditionable component of R, is rj, the "hope" (second 


ment) reaction; and it becomes connected to stimuli, 
fear doc 


4 produces a given 


Here the 
ary reinforce- 
5 & A, JURE d 
in punishment. The result is that whenever R; starts to occur, 
it is facilitated rather than blocked. This, it is sumed, is the essence 
of "habit," as opposed to "inhibition." Parenthetically, it is interesting 
to discover, from Websters Unabridged Dictionary, that "habit" and 
"inhibition" both come from the same Latin root, babare, meaning to 
have. It is good to have a theorctical conception of these two phe- 
nomena which sees them as closely related as the etymology of these 
words would thus imply. This is not the only oc $ 


wisdom has been found deeply imbedded 
language! 


) ion on which 
in the very structure of 


The second version of two-factor theory was "two-factored" in two 


a s 
SiR S S4———— —— R :s 
i f i 
s n) E frg) 
"yit Sp— Rp 
s s 
Sa Re Ss ah Si — Rs Sip 
gee s 


Fig. 8-4. Schematic representation of current 


two-factor theory. 
produced in behavior by reward, as well 


as d 
derived from conditioning plus the feedback 


Here the changes 
hose produced by punishment, are 
principle. 
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different ways: it distinguished between sign learning and solution 
learning did between incremental reinforcement and decremental 
being associated, presumably, with incre- 
solution learning with decremental rein- 
forcement. Now the present version of the theory is two-factored in 
only one way: namely, with respect to the two types of reinforcement, 
incremental and decremental. With respect to the other principle of 
ck sification employed in the second version, the theory is now de- 
cidedly onc-factored; that is, it assumes that all learning is sign learn- 
ing and that solution learning (as well as response inhibition) is a 
derivative thercof. 

In short, then, as far as “types 
vised version is "one-factored" 
gards type of reinforcement. Th 
for deriving a name for the theory; 
i perhaps serve as y 
kinds of reinforce: 


reinforcement, sign learning 
mental reinforcement and 


of learning" are concerned, the re- 
but it remains two-factored as re- 
is is perhaps not a very good basis 
but, for the time being at least, 
vell as any other, provided that 
ment but not two kinds 


this designation can [ 
it is taken as implying two 
of learning as well. 

ll. Early Adumbrations of the Present Position 

The debt of the present to the conceptions of E. L. 


Thorndike and |. P. Pavlov has already been indicated. Although 
Much time has beet ne how these conceptions have 
to be modified or be maximally useful, the fact 


PENTA rhat ehe principle is no less central to the 
emerging theoretical position th 
Carr (1938), MeGeoch (1942) 
the latter is virtually incontes able, at 


approach 


1 spent in suggestir 
refined. in order to 


of conditioning 
an is Thorndike’s Law of Effect. As 


and others have repeatedly observed, 
a grossly descriptive, empirical 
is completely incorporated into present thinking. 


In fact, Thorndike himself can even be said to have been a "two- 
factor” theorist of sorts; for the so-called Law of Fffect is really a Law 
of Effects, since it deals with rwo kinds of consequences w hich may 
Ir is true that Thorndike thought of these two 
in the sense of one making and the other unmak- 
ach being capable of producing its own 
as we shall see in Chapter 11, 
s views are correct even 


level and, as such, 


follow à given action. 
effects as reciprocal. 
ing habit "bonds." instead of € 
brand of “forward” learning. However, a5 
c in which Thorndike’ 
al departure here from the views of Thorn- 
account of some of the more immediate 
looked (i.e. various forms of sensory 


there is a limited sens 
i this respect. Our princi 
dike is that we take explicit 
effects of behavior which he over 
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feadbnele cf. Chapter 9) and incorporate, rather than o 
concept of conditioning, thus arriving at a notion of habit w 
supersedes his “bond” conception thereof. f MOON 
Pavlov, also, was a kind of early two-factor theorist. He ae kno us 
edged that some unconditioned stimuli (or reinforcers) arc diii: 
and others "negative"; and, as indicated in Chapter 3. he even i 
a preliminary effort to derive the phenomenon of "habit" from iniri 
the same basic principles as are emploved in the present agipraneh, 
The difficulty was that Pavlov was such an inveterate Objectivist that 


‘ ” s e m MED wey ich a 
he could not “see” certain of the intervening variables” which 
sophisticated behavior theory requires, 

There were, 


of course, even earlier adumbrations of the present 
position. For example, in the famous chapter on "Habit" in his Prin 
ciples of Psychology (1890), William James wrote as follows: 


In action grown habitual, what instigates each new mt ) (us 
to take place in its appointed order is not a thought or a perception, hoe AS 
sensation occasioned by tbe muscular contraction just finished. A strictly 
voluntary act has to be guided by idea, perception, and volution, throughout 
its whole course. In an habitual action, mere sensation is a sufficient guide, 
and the upper regions of the brain and mind are set comparatively free. A 
diagram | see Fig. 8-5] will make the matter clear: ~ 

Let 4, B, C D E, E G Tepresent an habitual chain of muscular contrac- 


tions, and let a, b, C, d, e, f stand for the respective sensations which these 
contractions excite in us when they are successively performed. Such sensa- 
tions will usually be of the muscl skin, or joints of the parts moved, but 
they may also be effects of the movement upon the eye or the car. T hrough 
them, and through them alone, we are made aware whether the contraction 


"This is not to say, of course, th 
adumbrations of the present view, 


lation. (cf. Chapter 2). But in the interest of time and 
Systematic effort is made in this volume to cite 
1890 and 1900 a whole new 
sort of natural hi 
definite 


scular contradiction 


at there were nor still earlier, wel earlier 
both in common sense and in scholarly specu- 
space conservation, no 
carlier thinking. Morcover, between 
“universe of discourse” 
atus between the past and 
“line of descent" 
"association. movement" 


was introduced which marks a 
the present. There is, however, a 
leading to contemporary learning theory from the 
as it appeared (primarily in 
preceding centuries—for the details of which see 
of Experimental Psychology (1929). 
"founder" of this movement, 
ideas" (cf. also Aristotle); and 
say today, a conditioned sens 


England) during the two 
à book such as Boring's History 
David Hartley (isos as the official 
held. thar learning Consists of the “association of 
an idea was, basically, 


ation (Boring, p. 
5). Thus the association of stimuli, 


condition for learning, is really 


an image or, as we might 
199; cf. also Mowrer, 1960, Chapter 
which since P; 
Just a more 


. ideas. 
other hand. was much more directly 


avlov has been taken as a crucial 
objective way of talking about the 
The connectionisin. of Thorndike, on the 


an outgrowth of the 
tarians (hedonists), such as Alexander Bain (1818 


association of sensations, images 


thinking of the utili- 
11903) (see Boring, pp. 223 ff). 
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has or has not occurred. When the series, 4, B, C, D, E, F, G, is being learned, 
ich of these sensations becomes the object of a separate perception by the 
ach movement, to sec if it be right before advancing to the 
next. We hesitate, compare, choose, review, reject, etc., by intellectual means; 
and the order by which the next movement is discharged is an expre order 
from the ideational centres after this deliberation has been gone through. 

In habitual action, on the contrary, the only impulse which the centres of 
d send down is the initial impulse, the command to 
2 (pp. 1157116). 


mind, By it we teste 


idea or perception nee 
start. This is represented in the diagram bs P A 


V 
Fig. 8-5. “Habitual chain of 
muscular contractions” (James, i á * é d " f 
1890, p. 116). For full expla- SNEEN N 
nation, see text. 
A B D E F G 


Cc i 


Whether James meant this analysis to apply only to a response 
chain, of more or less discrete acts, or thought of it also as germane to 
any given act or movement azhile in progress is not entirely clear. But 
this much is evident: he had a clear conception of the phenomenon 
of sensory feedback from on-going behavior as the cue and guide for 
further behavior. Our sumption is that behavior is thus guided by 
the hopes and fears W hich have been previously conditioned to these 

which produce, respectively, the right amount of 
tion of specific movements for the accomplish- 
“ James, of course, speaks more mentalistically, 


stimuli, emotions 
facilitation and inhibi 


ment of certain “acts Site à 
less “dynamically” about the process; but the similarity is obvious. 


In the latter part of Dewey's 1896 paper, on "The Reflex Arc Con- 
cept in Psychology SO there is an even closer approximation of present 
thinking. However. Dewey's comments on this score are discussed 
at engeh in another connection (Mowrer, 1960, Chapter 7—where 
furt ier reference is also made to James’ ideas) and will not be con- 


sidered here. 
Although 


Sigmund Freud spoke rarely of "learning" and appar- 
ently never of "learning theory.” vet his imprint on contemporary 
thought in this arca is unmistakable. In an era when organic theories 
of xersonality disturbance were verv much in vogue, Freud boldly 

1 holding that “neurosi far from being an "osis" 


challenged them by n ] 
fh lly learned (cf. Mowrer, 1960, Chapter 10). And 


of the nerves, is actua d ON o. C 
d into his systematic position certain ideas which have 


influenced subsequent developments in learning 
technical aspects. In holding that a neurotic “symp- 


he incorporate 
quite specifically 
theory in its more 
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tom" is essentially a way of "defending against" or "binding" anxiety, 
Freud (1936) definitely foreshadowed later work on avoidance lcarn- 
ing (see Chapters 2 and 3). And in his contention that much per- 
sonalitv change occurs on the basis of identification, Freud showed, 
by implication, the gross inadequacy and incompleteness of the then 
current “mechanical” learning schemata. As will be scen elsewhere 
(Mowrer, 1960, Chapter 3), Freudian thinking has directly stimulated 
recent laboratory work on imitation (especially as it relates to the 
development of language) which, in turn, has been highly important 
in fostering the general interpretation of habit which has been madc 
explicit in the preceding chapter. 

Freud's concepts of "cathexis" and "countercathexis" are definitely 
related to what we here designate as hope and fear; and even in re- 
spect to the role of sensory feedback from activities in progress there 
is a most interesting allusion. Writing on Freud and Hull in a recent 
issue of the American Scientist, McClelland (1957) put it thus: 


In the proposition which is basic to his whole system, 
the reduction in need automatically strengthens the connection between the 
stimulus and response preceding it, a statement of the well-known “law of 
effect,” which is usually thought to have been formulated first by Thorndike 
in this country. It is interesting to observe, however, that Freud, thinking 
along very similar lines, had come to an identical conclusion in the 1890's: 
“When this | the process of satisfying a desire] occurs, associations are forged 
- +. With the perceptual image of the satisfying object on the one | and, and 
on the other hand, the information derived from the motor activity that 
brought that object within reach” | Jones, 1953, P- 392]. In Hull's terms, he is 
saying that need reduction forges a connection between the “stimulus” and 
the response-produced stimulus, a conception quite similar to Hull's in most 
respects. It is probably because the basic “model” of how the organism func- 
tioned is so similar for the two men that it was so casy to incorporate certain 
Freudian notions into Hull's system. Both men were essentially natural 
scientists, who had been strongly influenced by the theory of evolution, 
and who tended, therefore, to think of the organism as responding and 


adapting to changes in the environment so that it could survive or maintain 
its equilibrium (pp. 108-109). 


Hull postulated that 


The important and salient point here 
formation derived from the motor 
"the response-produced stimulus." 
those immediate "effects" 


is the reference to “the in- 
activity” or, as McClelland says, 
Here Freud takes cognizance of 
of behavior which c 
overlooked, and relates them, albeit r. 


non of habit formation. Freud 


Thorndike consistently 
ather inexplicitly, to the phenome- 


+ as we have seen, also had the idea that 
when a stimulus is associated with a pleasurable state it takes on 


“positive cathexis^ and when associated with a painful state it takes on 
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if he had merely gone a step further and 
ced stimuli giving rise to positive and 
' he would have anticipated present theory 


"anticathexis." Therefore, 
had certain response-produ 
others to negative “cathexis,” 
quite perfectly. 
Hull also was aw 
from behavior and even went so far 
acts” (see Mowrer, 1960, Chapter 2), 1 
precisely. for their stimulus- or cue-producing b 
his aversion to including affective states in his formal 
Ü ossibility which here lay cle 


are of the stimulation which invariably results 
s to refer to “pure-stimulus 


acts which are performed 
aluc. But because of 
system, Hull 
arly within 


never made maximal use of the p 
his reach. 

A search of the literature between 
tionably reveal many other interesting 
ments in the ficld of learning theory; 
show that the position with which we 
broadly based and historically deeply rooted. 


1890 and 1920 would unques- 
forecasts of present develop- 
but those cited will suffice to 
are currently concerned is 


lll. Incentive Motivation: 
Hull, Crespi, and Spence 


It is one of those paradoxes in which the history of science so 


abounds that although Clark Hull took a leading role in c citing in- 
terest in what we have here termed ty pe-2 secondary reinforcement, 
he never fully saw its implications for general behavior theory. As has 
Hull repeatedly indicated his awareness of the 
ated stimulation; and he was also aware that 
uced stimuli can take on the kind 
"cathexis." But he never 
What happens when it is 


just been indicated, 
reality. of response-correl 
independent, environmentally 
which Fre 
asked the question, 
uli, rather than independent ones, that take 
on such a "charge" or, in his own terms, acquire the capacity to act 
as secondary reinforcers? It is, We assume, this phenomenon, coupled 
With that of fe r- mediated inhibition, \ hich shapes behavior into 
modes of action which are known as “habits” or, 


prod 


of “positive charge ud termec 


seems to have seriously 
response-correlated stim 


those characteristic 


equivalently, “voluntary acts . ; 
‘The nearest Hull came to this conception was In W hat he referred to 


as "incentive motivation" and which hc symbolized by the notation k. 
Spence (1951a, p ). one of Hull's chief expositors, has interpreted 
sta, p- 


this feature of his 


theoretical system thus: 

Incentive motivation (K). The properties of the reinforcing stimulus also 
e as 3 : £ 

a f the response being learned. In the case 


" " pe H 
appear to determine the strength O 
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of classical conditioning involving food as the US, and in instrumental lcarn- 
ing problems involving positive goal objects such as food, water, and so on, 
the magnitude of the reward has been shown to affect the response measure, 
for example, to lower or increase response latency, increase or decre se run- 
ning speed, and so forth (Crespi, 1944; Zeaman, 1949). Delay of the incentive 
has also been shown to determine the level of 
simple learning situation (Harker, 1950; Perin 
With these facts in mind, Hull h 
(K ) that is assumed to be 
of delay (Tg) of the rew 
be learned. 


performance attained in any 
«19432, 1943b; Perkins, 1947). 
as postulated an incentive motivational factor 
a function of the magnitude of (G) and the time 
ard following the occurrence of the response to 


K= FOV, To) 
In many Ways the incentive-motiy 


Hull's system (cf. Bolles, 1958, p. 109; and Bevan & Adamson, 1959. 


P- 2). Pavlov, much earlier, had referred to what he termed “natural” 
conditioned reflexes, by which he n 


ganisms to salivate or make other appropriate anticipatory response 
to the mere sight, or sound, or smell of objects such as food, sex 


partners, and other sources of primary gratification—unconditioncd 
stimuli he called them, 


These natural conditioned reflexes Pavlov con- 
sted to the artificial ones which are set up in the laboratory for 
experimental purposes. And, on a purely common-sense basis, one 
would expect a living organism to rcact, for example, to the sight of 
food in very much the same Way as to any other stimulus (c.g. a 
buzzer) which, by special arrangements, indicates that food is 
"near." And since a secondary reinforcer was for Hull essentially. the 
same as what Pavlov called a (positive) conditioned stimulus, it seems 
remarkable that Hull felt it necessary, in order to deal with the facts 
cited in the Spence quotation, to resort to the seemingly superfluous 
concept of “incentive motivation.” The pa 
from Hull's addiction to the idea that a secondary reinforcer, so- 
called, achieves its efficacy by strengthening some "habit" in the sense 
of a stimulus-response bond. ý ï 

Almost immediately after the appearance of Hull's first full-length 


statement of his system (1943), Crespi (1944) took strong exception 
to the wav in which Hull c 1 


; onceived and employed his K variable. 
He said: E 


ation concept was redundant in 


leant the tendency for living or- 


trouble, apparently, arose 
PI à 


The object of the present paper is to subject Hull's tre 
tive variation of incentive and performance 
the light of a recent series of experiments b 
the basis of this appraisal the writer will be 
conditioning explanation of the effects of i 
Suggested in its stead will be a theory g 
341). E 


| atment of quantita- 
to criticism and amplification in 
y the writer (Crespi, 1942). On 
led to reject Hull's strength-of- 
ncentive variation upon action. 
rounded upon emotional drive (p- 
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And later i i 
ater in aper Crespi ex s hi 
this paper Crespi expands his argument thus: 


The writer wi : 
ing User e oci idee that s rre drive has been vary- 
nve [in experiments. sited] " d Highs b» diterenr- amounts of incen- 
the Quo diertrmár cited], anc this factor rather than simple learning is 
incentive Mida of differences in levels of performance. With varying 
among thes pain after they have been experienced of course, there arise 
exciveinene E Dum of animals varying a mounts of anticipatory’ tension or 
Aree ionis ne proper of their acquisition. This is a differential of what 
ar Am E Enperia: The differential of eagerness, of emotional tension 
Bde n a ; d for the various amounts of incentive, generates, 
Cites oF. MS us N porne the differences within each trial in the obtained 
ik edom she sen The larger the incentive amount for which an animal 
ig nS ag the more cager he is to attain it, LC. the more emotional drive 
relent : ee his hunger drive to motivate his response. Eagerness is 
TAREA carning only in the sense that the animal must find out how much 
ve is obtaining before he exhibits a corresponding amount of eager- 


Ness ( p. ) 


35 


vay of thinking hardly 
as Crespi here employs 
(cf. also appetite, Chapter 
assuming that this 


M uy cie of all this to our present v 
s s emphasis. The concept of "c: 
alent to Popefuliess 
it. Crespi is similarly 
lo we; and he is further as- 


agernes 


"i MM equiv | 
Sess Nes have conceived 
Me iens is basically emotional, as d nd h 

g that the resulting modification in behavior 15 less a change 


in. “hab; a ; : 
y habit strength,” i.e. less a change in what the organism can do, 
an iti t : y Gee SNe 

In it is a change in what it avants to do. As Crespi puts it, 


and the preceding arguments make plaus- 
of incentive operate to influence 
ariation of emotional drive or antici- 
in levels of action . . . cannot be 
t strength or learning alone, but 
s in drive strength (p. 354). 


hie de onse tun just completed a 

Perfor pothesis that variations of 

patory ROS through the agency of v 

looked eagerness. . -> The differences in | 

Must [ upon as caused by differences in habi 
st be understood as owing 


amount 


also to difference 
is entirely congruent with the present 
t—a change or “difference in 
ts in intensified activity, 


This statement, as it stands, 
reinforcemen 


* Because it resul 
an increase in emotional or 


c also Section IV 
insistence that 


condary 


Conception. of : 
Ero drive strength.’ 
eas dona that this char 
dary drive; this, as we hav 
no means 
"gatur 
a major step forw 
asi-mathematical variable, K. 
ever to have assimilated completely 
theory of considerations such 
6) has taken up the 


c involves 
e seen (Chapter 6; 
certain; but Crespi's 
ıl secondary reinforcement” is an 
ard and a real advance 


ng 


of this chapter), is by 
a motivation" Or 
Over ee ag was 

ull's more indefinite, qu 
n. seems N 
atic behavior 
but Spence (195 


l 
tl Tull. as we have scc 
MS < " x 
1 implications for system 
as Agra à 
those advanced by Cre5ph 
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challenge and in his recently published Silliman Lectures, Pop a 
at Yale University, shows how profoundly shaken is Hull's sy semi 
when one comes to grips with the now available facts and angumënts. 
Hilgard (1956) has prepared an excellent précis of Spence's bords 
thinking, and we shall follow this in the present discussion. First we 
note that “the r;—s; mechanism [see Chapter 4, Section HI| becomes 
identified in Spence's theory with Hull's incentive motivation (K) 
rather than with habit strength (H)" (p. 416). This, as previously 
noted, is a thoroughly logical and long overduc development. And 
Spence goes on to replace Hull's definition of E (excitatory potential) 
with the simple formula: E = H(D +h), where D is drive. And 
Hilgard concludes his exposition thus: 


How can Spence’s theory best be characterized, in relation to other rein- 
forcement theories? Let us quote his own words: 

“This theory is, then, a reinforcement theory so fara 
is concerned; that is, the presence or absence of 
in its properties when present, do make a differ 
instrumental response. It is 
tional sense of the term, 
vary with variations in r 


excitatory potential 
a reinforcer, and differences 
renee in the strength of the 
not, however, a reinforcement theory in the tradi- 
for the habit or associative factor is not assumed to 
einforcement? (p. 419). 


The implications of this statement 
The statement is, first of all, 


bond conception of “habit, 
action potential” 


are numerous and far-reaching. 
an abandonment of the classical S—R 
“Tt says, in effect, that changes in “re- 
or, more simply, “performance” are jot dependent 
upon the previously assumed “strength of association” between stimu- 
lus and response but upon "the presence or absence of a reinforcer, 
by which is presumably meant a secondary reinforcer or "incentive" 
motivator.” 


This move has been foreshadowed for 
peatedly: expressed dissatisf. 
forcement. His chariness 
mitments has alre 


some time by Spence’s Fee 
action with the classical conception of rein- 
about making 
adv been noted (Ch 
reluctant to commit himself to 
occurs when 


any kind of ncurological com- 
apter 7); and he has been equally 
Hull's hypothesis that. reinforcement 


and only when a drive is reduced. For example, we read: 


# “Having decided (with H 


ull) that H is unaffecte 
reinforcement, the 


author (unlike Hull) considers th 
‘contiguity camp, at least insofar as ‘instrumental rew 
(This is a surprising dex clopment, since earlier in th 
of a ‘crucial experiment on the reinforcement 
ground that it is ‘Impossible to conduct 
motivation and reinforcing event)” 


d by magnitude and delay of 
at he [Spence] has joined the 
ard conditioning’ is concerned. 
he same lecture the possibility 
-contiguity issue’ is denied on the 


an experiment without some form of 
(Bitterman, 1957, p. 142). 
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ui Vac ui d of secondary reinforcement does not involve drive 
deon oe Tas don cla med that he is not a consistent. reinforcement 
E EY isenitiois) fails to note, however, that Hull does not abandon 
Chine ement position in the case of secondary reinforcement, but only 

special hypothesis that reinforcement aly involves drive reduction” 


(Spence, 1951b, p. 709). 


But if the term “reinforcement” is not to be purely circular, L.e., 
equated to learning itself and thus incapable of “explaining” learning, 
nome independent criterion thereof must be specified. If it is not drive 
reduction, then, what is it? If the definition of reinforcement is left 


indefinite—Spence has often spoken of himself as accepting "a theory 
Of general reinforcement’ —it can hardly avoid being circular. How- 
ever, by means of the radically new position, Spence, by means which 
Will presently be indicated, at last escapes from this dilemma. 

What the new view says is that performance changes as a result 
of change, not in S—R bonds, but as a result of changes in the incentive 
(secondary-reinforeement) value of particular stimuli. And that is also 
the View which we are here espousing. There is, however, this salient 
divergence, As indicated in a passage quoted earlier in this book 
(Chapter 4). Spence takes the postion that “the K factors (based on 
anticipatory goal response) act 35 drive" (Hilgard, 1956. p. +17); 
Whereas our supposition is that an incentive or secondary reinforcer 
i, basically, a stimulus which mediates à reduction in (secondary) 
drive, Although the latter view is not without complications which 

F resolved (see Chapter 6), they seem less 
ion that secondary 


on the assumpt 
mechanism,” is a form of 


a: 


T only tentatively 
us than do those attendant up 
as the “Tesk 
Recently, on 
atter thus: 
ised in theory 
actional anticipatory goal re- 
1 secondary rcinforcement. 


an examination an ad- 
“This intervening varia- 
in a way which is 


qu nforcement, conceived 
eibi increase or excitation. 

sed graduate student put the m 
le is hard to index, and again it is u 
Not particularly clear—€-g- the same fr: 


s A "m 
Ponse explains both secondary drive ae mone 
a behavior sequence the drive function ends 


tever : 
, Never know where in : : : 
and the reward function begins.” The conception of secondary rein- 
forcement (type-2) developed in Chapter $ and given special applica- 
tion to habit formation in Chapter 7; whatever its other weaknesses 


hay be, does not suffer from this form of ambiguity. 
mined by the "vigor of the ‘classically conditioned’ 
i regarded as dts underlying ‘mechanism.’ 
sretation of reinforcement... )7 


ib) equating of the r, mechanis 
5 Li n 


wees 
la ee of K is deter 
Cheat oal-response, Tos 
SR Sony thus is tied to a co 
sid et d 1957. P- 142). See 

Secondary reinforcement. 


which is 
nsummatory interp 
also Spence's (195 
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Hilgard also points out the following interesting development. in 
Spence’s thinking: 


Because reinforcement may be necessary for classical conditioning and is 
not decisive for instrumental learning, Spence toys with the notion (without 
committing himself to it) that he may have a two-factor theory just. the 
opposite of Mowrer's: 

“It is interesting to note that if one were to adopt a theory that reinforce- 
ment (e.g., need reduction) plays a decisive role in the acquisition of habit 
strength of classical conditioned responses but not in the case of instru- 
mental conditioned responses, one would have a two-factor the 
exactly the opposite of the well known two-factor theory 
Schlosberg, Mowrer, and others” (Pya 


ory that is 
espoused by 


Implicit here is the assumption that incentive motivators or sec- 
ondary reinforcers acquire their potency as a consequence of tem- 
poral contiguity with primary drive reduction. This, too, is our as- 
sumption (see Chapter 5). And it is our further assumption that so- 
called. "instrumental" conditioning or habit formation is dependent 
upon this kind of conditioning (Chapter 7). But instead of this state 
of affairs implying the reversal of the two-factor position. (version 
two) which Spence suggests, it, more logically, implies (a) that all 
learning is in the nature of “classical conditioning” and that so-called 
instrumental conditioning is derived thereform and (b) that there 
are, however, two kinds of classical conditioning: that which occurs 
when a drive is reduced (decremental reinforcement) and that which 
occurs when a drive is induced (incremental reinforcement). Spence's 
system, as it presently stands is, like Hull's, seriously deficient in its 
lack of reference to the latter fact.^ j 


Incidentally, in passing it is interesting to note the following ex- 
cerpt from Hilgard’s résumé: 


Spence indicates two reasons why, over the years, he was not actively in- 
terested in the mechanism of reinforcement, The first was his interest in 
quantification, which he repeatedly pointed out did not require a theory of 

' “Reluctance to renounce the 
ditioning leads then to the tentativ 
position—'that reinforcement (c 
which do not seem to have 
Pop?) 

* "Spence repeatedly emph: 
within which his theory 
the book passing refe 1 


necessity of reinforcement. for classical con- 
€ statement of a kind of inverted ‘two-factor 
g- need reduction) the implications of 
been thought through very clearly". (Bitterman, 1957, 


asizes the narrowness of the 
is intended to apply 
rence is made to a trout 


‘boundary conditions’ 
XE Although here and there in 


lesome fact, the uninitiated reader 
at the overall empirical fit of the 


an the lack of it” (Bitterm: gs 
than à f : Bitterman, 1957, p. 144). Tt must be admitted that although 
some of Birterman's structures Upon Spence's views 


charge of “narrowness” would be less well Justified 


might never guess th j 
P - theory is far less impressive 


also apply to our own, the 
Pp^ 
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reinforcement. T he second was the difficulty that showed up in experiments 
on latent learning of ever finding an experimental situation totally free of 
Tort to explain the various things happening 


reinforcing effects. It was the et 
at led to an interpretation of reinforcement 


in latent-learning experiments th 
important to Spence's contemporary theorizing (pp. 415-16). 

As indicated elsewhere (Mowrer, 1960, Chapter 2), our present 
arning meshes nicely with the facts of so-called latent 
1 that as Spence's thinking develops along 
of congruence which is already evident 


conception of lc: 
learning; and it is presumec 
these new lines, the degree 
will become greater still. 


IV. Seward on “Tertiary Motivation" — 
and a Recapitulation 
confluence. of contemporary thinking with 


vey » B : -—i na D d “ar, 
respect to the psychology of learning is the work of J. P. Sew ard. 
As carly as 1950 ‘this writer had moved a long way toward the posi- 


under discussion. 
ossibilities suggested by Wolpe 
term [reinforcement] 
does not necessarily 
an simply that when 
kelv to occur. As we shall see, 
à ability of a response 


Also indicative of the 


ton which is presently He said: 
Vhere is a third possible alternative [to poss! I 

(1950) and by Meehl (1950) | that docs not confine Uf 

to the learning process. Increase 1n response strength 

Mean increase in strength of S—R connections; It Be 

3 situation is repeated. the response Is mo i rol 

there are other wavs than learning by which the prob 

May ; 

Uy be increased .. . (p 363): 

What Seward meant by this provocative statement becomes clear 


fr : ; : f 

rom the following discussion: 

The question b fore us is er increases response strength, 

Does it do so by strengthening S—R connections Für future use, or does it 

help on that fatte occasion to bring about the response: In amer words we 

are facing that much debated issue, iS reinforcement a matter of learning or 
g th a 


Performance? (P 365): 


how a rcinforc 


And then Seward says: 
c following nutshell: A reinforcer, in the 


rey through its ability, native or ac- 
ise in the presence of the relevant 
d to stimuli associated 


restated in th 
s its poter 
1 response ns 
s conditione 
cse stimuli recur they there- 


acilitates the act in progress 


em c Hinury may be r c 
quir A positive cathexis, gain 
nee Cd, to arouse à consummarory te à 
ced. In a learning situation this become 
Path the preceding instrumental. act- W hen th 
Ore arouse tertiary motivation, which in turn f 


(P. 372, italics added). 


al 


he foregoing is Seward's very explicit 
ed with the preceding 


in tl ) 
associat 


D TE] a 
Particularly noteworthy foe 
“stimull 


reference to the role. which 
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instrument act” (so-called response-produced or Lr ed 
stimuli) presumably plav in the development. of a habit. den ced, 
his formulation differs from the present onc only in onc regard: Ux 
stead of positing that it is "secondary reinforcement that gets epe 
tioned to such stimuli (in the process of habit formation), Sew ma 
refers to what he called “tertiary motivation." © What does this imply? 
In a footnote Seward gives this important part of the answer: 


Although the definition of tertiary motiv 
facilitation of a goal set, is confined to positive demands, a three-component 
theory of motivation is intended to apply to aversion as well. ‘I hus pain d 
be considered a primary drive, fear a secondary dr and anticipated fear 


reduction (or a fractional antedating escape reaction) a tertiary drive (p. 
370). 


ation here proposed, as duc to 


Here, obviously, what we have called hope Seward is calling tertiary 
Motivation and tertiary drive. But whereas our system requires us to 
think of hope (“anticipated fear reduction" in Seward's terms) as 
itself involving secondary-drive reduction, Seward sees a drive or 
Motivation increment in the situation. This, obviously, is the heart of 
an important matter. We have already seen (Chapter 6) that sub- 
jectively there is considerable evidence for believing thar at least dd 
a particular stage of "hoping" there may indeed be an increment in 
motivation of the kind commonly termed appetite; but considera- 
tions previously advanced suggest that appetite is a special aspect of the 
more gencral problem of hope and that we obscure the larger truth 


if we focus on the drive-increment rather than the drive-decrement 
aspect of such situations. 


There is, of course, the thorou 
appearance of 
motivating, 
of the subjec 


ghly objective observation that the 
a stimulus with secondary-reinforcing, or “tertiary 
Properties often results in 


augmented activity on the part 
t. What better evidence, 


one might ask, of a correspond- 
"Tt is possible th 


at more is here being 
than is legitimate, 


It may be that by 
mental act^ he means, not re 


merely those stimuli 


“read into" 


Ccward's 1950 statements 
"stimuli associated with the preceding instru- 
sponse-correlated or response-produced stimuli, but 
re present, more or less incidentally, when the 
ated and occurs, If this is truc, then Seward's position 
articular respect, more similiar to that of Hull Cor perhaps 
Guthrie, see Section VIII; than to the Present formulation, Seward nevertheless 
makes statements such as this: “When these stimuli recur they therefore arouse 
Which in turn facilitates the act in progress” 


added), which can certainly be interpreted. as referring to stimuli which arc, in 
fact, produced by "the act in progress.” [f th, 


parallelism is great (sec also Chap: 


Which 
response is (otherwise) instig 
is, in at least this F 


tertiary motivation, italics 


(p. 3 


is is the intended meaning, then the 
erg). 
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ing increase in "motivation"? But attention has already been called to 
the fact that motivation and motility are by no means perfectly cor- 
related; and still further evidence on this score will be taken up early 
in the next section. In the meantime, a few further words will bring 
Seward's thinking in this area more fully up to date. In a paper pub- 
lished in 1956 this writer makes a number of assumptions which are 
completely congruent with the present analysis. He posits that “mediat- 
Ing or response-produced stimuli" may have conditioned to them 
“catisfiers.” The former are conditioned when 
ain disturbers of 


either “instigators” or 
response-produced stimuli are followed “by cer 
i ternal (c.g. pain and pain signals) or internal 
are conditioned when re- 


homeostasis, whether ex 
(e.g, metabolic deficit),” and the latter 
sponsc-produced stimuli are followed “by removal of the disturbing 
stimulus or bv replacement of the deficit” (1956b, p. 200). This 
language corresponds to our assumption that, as a result of being fol- 
lowed by cither primary punishment or primary reward, response- 
on the capacity to arouse, respectively, fear or 
al forecasters of things to come then guide 
the subject, more or less reliably, away from actions (or places) that 
are likely to be harmful and into actions (or places) that are likely to be 
helpful. For this process, commonly but not entirely appropriately 
termed “habit formation,” Seward suggests the very apt term, response 
selection. 


Only in one respect is the Sewar 
rhaps more accurately, ours different 


admits to being “on the fence": 


correlated stimuli take 
hope and that these emotion: 


d position importantly different 


from our present one (or, pe 
from his)—and even here Seward 
My own attempts to solve this problem, starting five or ten years ago, re- 
flect its stubbornness. In retrospect 1 find I have vacillated between the two 
Viewpoints here brought face to face—between considering | secondary? ] 
tisfiers and as incentives (p. 199). 
favor the latter possiblity; but he himself 
ement of theory.” and perhaps 


ities Will still further dissolve 


rewards as s 


Seward still seems to l 
refers to his formulation as “only a fr 
explicit the dissimil: S 
Seward has been an indefatigable investigator and 
; and much of the movement in recent 


as it becomes more 
themselves. Certainly 


an intrepid thinker in this area 
vears toward unifications has undoubtedly been a reflection of his 


experimental ingenuity, logical astuteness, and complete candor and 


fair-mindedness. 
This and the prece 


As we have seen, there is today 
andon the S—R bond conception of habit and to 


ding sections can now be summarized as follows. 
a decided tendency among major learn- 


ing theorists to ab 
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stress instead the notion of “incentive (terti ry) motivation.” But we 
are now in a transitional period, with some writers holding that, be- 
cause incentives "produce" activity, they must involve motivation in- 
crease; whereas others incline to the view that incentives (secondary 
reinforcers) achieve their effects upon behavior through a decrease in 
motivation (conceived as pure drive rather than an “urge to act"). 
Take the question of why human beings drink alcoholic beverages, 
ie, why they engage in responses which lead to their becoming 
intoxicated (in whatever degree). The present analysis suggests that 
when an individual drinks he does so, not just because he is tense, 
anxious, depressed, but because when he merely thinks of drinking, 
his tensions drop somewhat—and come right back up again when he 
thinks of zot drinking! It is, we hold, this token relief of tension that 
“drives” the individual to drink—and to do all the other things he 
“habitually” does.? It seems most unlikely that a person would “ha- 
bitually” drink if each time he thought of it he got more tense. In 
fact, one well-known method of treating alcoholism consists of dc- 


liberately conditioning a specific drive (nausea) to the stimuli as- 


sociated with alcohol consumption. The only way, apparently, that onc 
can consistently hold to ¢ 


a a drive-increment conception of the role of 
“incentives” is to assume that some forms of drive are inherently 
“pleasant,” whereas most are admitted] 
many difficulties, 
dependent ( 


y not. This position involves 
not the least of which is that it provides no in- 
and noncircular) criterion of w hat drives are and are not 
"pleasant." All things considered, a thoroughgoing homeostatic posi- 
uon seems most defensible here.” 

Some readers may be troubled by the 
in the foregoing discussion. c 
are teleological, and frank 
by its anticipated ends anc 
ism which medi 


appearances of teleology 
From one point of view our assumptions 
ly so. We assume that behavior is governed 
l consequences, But the underlying mechan- 
| ates the process is thoroughly cause-and-effect. Our 
hypothesis says, quite simply, that the alcoholic drinks because he is 
"drunk." In other words, because thc 
duces some of the effects of liquor 
propelled toward consumm 


anticipation of drinking pro- 
itself, the alcoholic is powerfully 


ation of the act. The anticipatory satisfac- 


See Lillian Roth's, Pi Cry Tomorrow (1954), for a highly sensitive and illumi- 
nating account of the psychology of alcoholism, especially her discussion of what 
just the thought of having or not having alcohol available means to an alcoholic. 


See also Hull's C1931) paper on “Goal Attraction and Dirceting Ideas Conecived as 
Habit Phenomena. f 
“For a survey of empirical studies on * 


3 “incentive motivation” which cannot be 
discussed here, see M 


ason (1956) and Dyal (1957) 
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tion is, of course, short-lived unless the individual actually 770ves for- 
ward in the sequence of actions which on past occasions has led to 
drinking. If, at any point in this sequence, the individual stops or turns 
about, then his anxiety and despair return in full force; and this is 
ard (activated) by what is effectively a 


how it is that he is led forw 
rather than increase in motivation, He, like "habituated" 
is, properly speaking, driven by the tension and 
and is, simultaneously, tempted by the 
head. With, so to say, one’s 
one direction in which one 


reduction 
organisms in general, 
discomfort that lie "behind" 
“path of least resistance". which lies a 
“back against the wall," there is only 


can move. 
Thus are we able to resolve the dilemma posed by what Seward has 


called “tertiary motivation" or, alternatively, the “motivating | rather 
than relaxing]. function of reward” (Seward, 1951). Reward brings 
. and hope leads to action. This way of thinking 


hope into existence 
will be developed further in the next section. 
versus “Reinforcement”: 


V. “Motivation” 
cClelland and Collaborators 


The Views of M 


esearches and theorizing of P. T. Young 
1 some detail, and will be briefly con- 
In the meantime it will be instructive to review 
advanced by McClelland, Atkinson, Clark, & 
Lowell (1953). 5 hich has recently attracted much attention and 
interest. In mans respects this system is remarkably similar to the one 
under discussion. Reference has already been made (Chapter 5, 
Section VII) to the virtual identity between the “ecology of emotions” 
derived by these investigators and the one developed in preceding 

A Clelland ef al. are likewise skeptical of the assumption 


chapters. And Mc $ 
that drive, in the sense of organic need or discomfort, can necessarily 
be inferred from an organism's level of overt activity. They say: 
The idea that motives are csesntia lly “tensional” in nature and energize 
organisms certainly has its difficulties which vary somewhat according to 
the way in which the tension is conceived. Hebb (1949) has objected vigor- 
ously and effectively to the notion of a motive as an energizer. . . . The same 
point has been made by Brown and Jacobs (1949). who state that one should 
abandon “the rather limited assumption that drives, when functioning as 
energizers, always lead to more vigorous overt or random action" (pp. 8-9). 


And by wav of making the point that a drop in drive may actually 
release rather than depress behavior, these same writers go on to 


quote from Hebb 


In Chapter 6 some of the r s 
have already been alluded to, 1 
sidered again presently. 
a systematic position, 


as follows: 
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“A behavioral excitation, an in c in some bodily activity, is dl = 
sarily a sign of increased neural activity either in the brain a s bah nay 
some part of it. The point is well illustrated by the process of gettir peme 
A small amount of alcohol may be an excitant—socially, and in its Wwe à : 
net effect on behavior—but this, of course, does not prove that alcohol is à 
neural excitant" (p. 209). 


In concluding their discussion of this issue, McClelland et al. sav: 


Thus the idea behind the stimulus intensity model, that strong afferent 
stimulation leads to strong, central neural activity w hich leads to strong 
response activity, appears to be much too simple. soc We are nnt edger 
that there is 0 connection between amount of activity and motive level au 
only that the relationship is complex and that therefore some other measure 
of motive strength may be preferable. It is perhaps because of such con- 
siderations that Miller (1951b), as well as Brown and Jacobs, argues for mer 
ing, rather than increased activity, as the chief criterion for inferring the 
existence of a drive (p. 21). 

The relevance of all this is, first of all, th 
inference that because a secondary 
releases activity it necessarily 
(see preceding sections). If, 


at it calls into question the 
reinforcer or incentive stimulus 
involves stimulation or drive increase 
as Hebb observes, alcohol (which is 
chemically related to ether and which likewise has anesthetic proper- 
ties) can, in the process of relaxing the 


person, also heighten activity. 
there is nothing paradoxical 


about the assumption that an organism 


is most likely. to go into action when it is hopeful, which is to say, 


under somewhat Jess drive or tension than whe 
is that an anticipation of more basic 
what satisfying may 


n hopeless. "The point 
satisfaction which itself is some- 

release, organize, or focus behavior and thus 
give the appearance of “increased drive" w hen, in fact, it is just the 
reverse phenomenon. The trouble, apparently, is that in both common 
speech and in more scientific discourse we do not always distinguish 
carefully between drive (as pure discomfort) and “drive” 
industry, organization, busy-ness. 
whether the individual has 


the “go-ahead,” 


as activity, 
The key to the latter connotation is 
as “received the word," ie, 


gotten the cle, 
or is unfocused, aimless, 


hopeless.” 


of the revolt against primitive S—R psychology 1s 


puting the assumption of a one-to-one 
The following excerpts are typical. “I he 


ause on some occasions an animal will respond in a 
on other occasions it fails to respond? (J 


makes it very clear that merely because a stimulus is phy: 
intense is no assurance thar it will raise the drive level... 


same point that Skinner ( 1938) is making when he savs th 
(Malmo, (958. p. 230). "It may be because 


“Indicative of the generality 
the number of writers who 


are now dis 
relationship between drive 


and a 
problem of drive arises bee 


given situation while 
"Brown's review 


tivity. 


Bolles, r958, p. 22)+ 


This is probably the 
at drive is not a stimulus 
| have been steeped in ‘drive’ psy- 
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MeClelland et al. then make an inference which, if the present 
writer fully unders ands it, is most intriguing. "They assume, in effect, 
that the so-called. primary drives are not motives. "In our ystem," 
they sav, "all motives are learned" (p. 28). And by this they mean, 
apparently, that a primary state of drive or tension leaves the organism 
essentially: inert, until and unless certain “affective states” have become 
conditioned to independent or response-correlated stimuli. These 
stimuli then become capable of “affective arousal,” which brings 
motivation and activity into being (cf. Skinner, 1958, p- gs). 

Psychologists have for some time now followed the practice of 
speaking of primary and secondary motives and reinforcers as if they 
functioned in essentially the same ways, except that the latter are 
acquired and the former are innate. Perhaps the time has come to make 
a more radical distinction. If, as now seems likely, there is no necessary 
relationship between intensity of primary drive and overt activity, 
ale to speak of hopes, fears and other 
* in the sense of controlling, organiz- 
c time we know that the 


then it may indeed be desi 
emotions as the truc “motivators,” 
ing, and directing behavior. But at the sam 
so-called primary-drive states in some way important, and a 
moment's reflection reminds us th 
on the basis of which our specific hopes 
learned. Hence we may say that the biologic ; 
of primary-drive increments and decrements provide 
acquisition of those behavior selectors which we called fears and hopes; 
but, strictly speaking. only these emotions (Or affects) motivate, 
Whereas primary-drive and decrements reinforce, 1C. 


mediate learning. 

Only one complication 
tween (secondary) motiv 
we thus conceive the motiv 


are 
at they serve as primary “reinforcers,” 
and fears are themselves 
ally all-important events 
the basis for the 


increments 
arises here, in making this distinction. be- 
ators and (primary) rcinforcers: Although 


ation function in such a way as to restrict it 


drive concept as the current villain of 


this warfare among dynamic schools. of thought. ] hope u 2 I write me 
off as a veneti anas if 1 answer my initial question, Are there drives? in the nega- 
tive" (Monroe, 1956, p» 25) “Although these experiments [by Campbell & Shef- 
field, 1953, and Sheffield & Campbell, 19s4—see Chapter 6. p. 198] show that 
drives do not inevitably goad the animal. into activity, they do not rule out the 
possibility that drives “are strong stimuli. LI Ir will be remembered. that my 
definition of drives says nothing about activity, but rather deals with increasing 


the performance of responses 7 arded by the offset of the drive or by the goal 
à i (Miller, 19592. p- 255)+ Here, in different words, 


ed in the text above: that drives may 


o consider the 


chologies that I have begun © 


objects that produce satiation 


Miller is making the same pont 
have reinforcing functions without neces: 


as Is st 
sarily acting as “motives.” 
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T T and sce the primary drives solely as reinforcers, Mi Du 
is that the emotions can also act as reinforcers, in the sene ut pi ov ye 
the basis for higher-order conditioning. But aside from this, the Pai i 
tion seems to stand. So the conclusion with which one would so 
emerge would go like this: primary (biologically given) dris es "d 
not directly "motivate" behavior at all—their chief function is to pite 
vide the basis for the conditioning of hopes and fears to Vi apes i 
response-dependent stimuli; whereas the hopes and fears thus acquired 
have a double function—they motivate (direct) behavior and also 
provide the basis for further (higher-order) conditioning. 
Another way of putting this same gener 
that the primary drives cannot “motiv 
directing it because thev, 
ditioned to 


al point of view is to say 
ate” behavior in the sense of 


themselves, are not capable of being con- 
either independent. or response 
other words, Moving toward or 
ing or inhibiting a particular 
organism feels; but such beh 
thirst, 


-dependent stimuli. In 
away from a given object or perform- 
act can alter the hope or fear that an 


avior will not, of itself, cause hunger, 
or any other primary drive to vary in 
why it was previously remarked that the 
"inert," nondirective, 
functions, as expl 


a similar way. This is 
primary drives are essentially 
nonmotivating. They serve only reinforcing 
ained above. This way of thinking, obviously, can 
emerge only when one has abandoned the S—R connection notion of 
habit and embraced the feedback idea. 
terms of S—R bonds, 


“motivate,” 


As long as one is thinking 1n 
then any stimulus, any drive can be said to 


OF cause, a particular re ponse.” But when one adopts the 
view that we behave as we do because we 


get from “regions” of emotional tension to 
comfort, then we see that the prim: 
immediately, “motivating” 
velopment of our hopes 
or reinforcement, alre 


are constantly trying tO 
“regions” of emotional 
ry drives are not, directly and 
but are responsible for the original de- 
and fears, in terms of the principles of learning. 


ady delineated, 


w A ; 
T Fe ; ; : » ; : ree n sc 

Here Another comment by Miller (19593) is apposite: "Srimulus-respon 
psychologists have concentrated on determining the 
tions of response to stimuli. s 


laws governing the connec- 
to 


: ; » said 
Thus, Sumulus-response psychologists may be sar 


know and care relatively. little about either stimuli or responses; they are 


specialists at. the hyphen between the capital S and capital R and could more 
aptly be called “hyphen psychologists, or to use Thorndike’s term, 'connectionists- 
While admitting our igi l trust ir iy Nor necessary to point out that we 
have made considerable progress on this basis. Unfortunately it is also true that 
other theorists are not far ahead of us in solving these particular problems” p 
242; sec also Lashley, 1950). 


Norance, 
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‘That this conclusion is congruent with the views of McClelland and 


collaborators is indicated by the following quotation: 


ase motives on affective arousal, following Young’s 
sons. In the first place, it seems apparent that the 
motive concept will be useful only if it has some kind of limited base. That 
is, if all associations are motivating, then there seems no particular reason 
to introduce the concept of motivation to apply to a particular subclass of 
associations. And we have chosen affective states as the basis for motives 
ds or strong stimuli because of the limitations of 
A more positive reason for choosing affec- 
tive states as primary is that they “obviously” important in controlling 
behavior, at least at the common-sense level. The hedonic or pleasure-pain 
view of motivation is certainly one of the oldest in psychological thinking 
and can be traced at least to Plato’s Protagoras. Furthermore, in order to get 
Motives in the laboratory we commonly pair cues with affective states re- 
sulting from shock, saccharine in the mouth, food deprivation, and the like. 
Operationally we manipulate states which we know subjectively will pro- 
duce pleasure and pain when we work with motives (p. 30). 


We have decided to ba 
lead ( 1949b), for several re 


rather than biological nec 
those concepts already discussed. 


VI. Hedonism or Homeostasis? 


The Views of Young and McClelland Compared 


eward in Section IV of this chapter begins 
w of Effect as originally formulated bv 
same time both hedonistic ad homeo- 


The paper last cited by S 
v pointing out that the La 
Thorndike was at one and the 


static. Initially Thorndike spoke of 
thus. implyir 


“satisfiers” (rewards) and "an- 
ng both the pleasure-pain re- 
reduction and drive induction. 
times to be more behavioristic 
terminology; and the 


noyers” (punishments), impl 
ationship and the dimension of drive 
But Thorndike, under pressure of the i 
(and less subjective), abandoned his former 
Law of Effect has since. been thought of mainly in the context of 
al ( physiological) gain and loss, i.c., In terms of the homeostatic 
í nplies that pleasure (if this term be used) is correlated 
tion—and pain, with drive induction. 

ping with an older precedent) has insisted 


nNologi 


model, which it 
with drive reduc 

However, Young (in keep 
“hedonic” ¢ 
be inherently pleasant or unpleasant, agreeable 


reference to any necessary correlation with 
And here he cites particularly the case 
he insists (and with some reason), are 
and of controlling behavior. 


upon a purely lement in the picture, i.e.. the tendency 
for some stimuli to 
or disagrecable, without 
drive reduction or induction. 
of tastes and odors. They too. s 
capable of producing "affective arousal 


But how, pre isclv, can this occur? 
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Perhaps the most promising answer is one that deriv " from nos 
siderations already suggested in Chapter 6. Living DM 
apparently constructed so that some foods will taste good bd Pis 
because it is such a long way (both phy sically and temporally’) " 
tween ingestion and digestion. The ultimate benefit (or harm) Xu 
come only from digestion; but in order to bridge this gap. little 
“sentinels” are put in the mouth and nose which predict, NH d 
reliability but by no means perfectly, what will bc good for us is 
what will not be. Later (in Chapter 10) we shall see how uscfu 
learned affects (of fear or hope) can be in bridging the gap between 
instrumentally: appropriate behavior and delaye 
punishment. Now in connection with tastes : : 
other forms of sensory stimulation), it seems that there arc innately 
given, built-in provisions for “affective arousal” which function X 
a similar way (as “warnings” or “approvals” ) whenever learning woulc 


m E “instinctive” provisions 
be too slow and hazardous. As indicated, such “instinctive provisio 
are by no means perfect: some 


good and be biologically FRU 
substances may taste bad but be very good for us, as in the case, for 
example, of certain medicines. However, the 
a valid one between how things t 
us physiologically, And it also 
induction provide 


d primary reward or 
and odors (and certain 


substances (e.g., saccharine) may taste 
useless or even poisonous, whereas other 


correlation is, in general, 
aste and smell and how they affect 
Appears that just as drive reduction and 
one basis for “affective 
so also may what Young [5 
In other words, the mere c 
out reducing a 


arousal” and conditioning, 
alls palatability provide another such basis. 
Xperience of something 
drive) can a 

ditioning of secondary 
experience of somethir 


for the conditioning 


ting good (with- 
pparently provide the basis for the con- 
reinforcement (or hope); and the mere 
ng tasting bad can simil: 

of secondary 
disgust), without any incre i 


Thus, rather than secing 
and homeostasis, we see th 
but with hedonism, 
homeostasis 


5 


rly provide the b 
Motivation (or fear, aversion, 
ase in the primary 


y hunger drive. 
any 


Necessary opposition between hedonism 
at both mech 
in the final an 
The present writer's 
evolutionary scheme 
value, only if 


anisms are real and important, 
alysis, being the handmaiden of 
assumption is that in the broad 
of things, pleasure is important, i.e. has survival 


it is correlated with homeostatic adjustment. Somewhat 
inexplicably, McClelland and Young take the reverse position that 


drive decrement and increment are but “special cases of a more gencral 
theory” (McClelland et al., p- Se), 


Perhaps sexuality will provide 
it seems to offer the best c 


an instructive example here. Certainly 


ase for hedonism, for sexual gratification 1s 
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an individual can apparently 
without sexual gratification 


See ee physiologically necessary, ines, 
n SE ME (perhaps even longer) 
ee ims ve the homeostatic model seems to break down com- 
M Fd . taking a larger view, we sce homeostasis still at work. In 
"m ur is sexuality it is the survival of the race, rather than of 
eens ; n: hich is at stake; and because the delay between relevant 
Wide m the biological result is here not a matter of seconds or 

s (as in the case of food ingestion and assimilation) but of 
“homeostatic™ mechanism would work 
a hedonic one. But the ultimate 
homeostatic, survivalistic.!! 
] the learning ssociated 
pure homeostasis. Here drive induc- 
and pleasure are perfectly cor- 
c is sexuality, where the hedonic 
iscernible only as à background 


GENA or months, no simple 
DPE be supplemented by i 

i ation, it seems, is still biological, 
oe one extreme. electric shock anc 
os e provides an example of 
S c See and drive reduction 
princi oe nd at the opposite extrem 
MUR n Is striking and homeostasis di: aibe ated 
enan there is no basic incompatibility- we aS es for 
cent EN ated stimuli to acquire affective volume ue primary 
biol i eenen (type-I. or type-D). this occurs; but where there i5 

ogical hazard in such an arrangement, learning scems to occur also 
^ the basis of a hedonic mechanism of the kind des -ribed. We are still 
ar from knowing how to talk about these matters with satisfactory 
Precision; but even the present state of uncertainty here is an advance 
over former certainty born of gross over-simplification of the kind 
Involved. in primitive stimulus-response psychology and in classical 


hedonism, 


as Conceived by 


VII. n 
- "Sensory Integration" 
h & Bitterman 


Woodworth and by Birc 
only drawn t 
ment, (b) 


forms of learning 


etw een threc 
„and (c) S—R 


A distinction is comm à ae 
S—s contiguity 


theory: i Lá 
ry: (a) S—R reinforce 
it can also operate in a 
world population of 
verse, as 


situations, 
with the 
sm operating in rev 
of the pow erful immediate pleasure 
out-compete the very legitimate, 
al world- want and —literally—starvation, 
bers and. alas. equally constant decrease in 
her species have, in the past. multiplied them- 
reason—except reason (sec. Mowrer, 


us (cf. Huxley. 1958). 


is in some 
sent time. 
s mechant 


Wye 
Useful as such a mechanism 
At the pre 
"owe see thi 
ned. Because 
a tendency to 
wide 


distinctly 

iun netly maladaptive way 

far an beings fairly “exploding, 
] is concert 


as biological surviva 
it has 


m" 
M ee with the sexual act, 
leise concern W ith e 
natural " the constant jnerease 
selves Tesources. We know that other | 
1960, D out of business: and there is no 
» Chapter 5)—why it should not happen te 


ventu: 
in num 
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contiguity (Spence, :951a; Seward, 1956a). There is, evidently, a 
growing conviction in the field that none of these conceptualiz tions, 
alone, is entirely adequate and that in order to match reality something 
of a composite theory is needed. This is what we are presently attempt- 
ing to develop. 


A major divi 
with the 
Thus one 


sion of opinion among learning theorists is the one concerned 
conception of the nature of the hy pothetical learning pon 
group of theorists, the associationists CI horndike, 1935; Hull, 
1943; Guthrie, 1935) has employed the notion of a functional connection of 
some kind of linkage becoming gradu 


ally established or strengthened with 
successive practice oc 


ions, Within this point of view one may find differ- 
ent preferences with respect to whether phy siological concepts (for RS 
receptor-effector connections in the nervous system) or nonphysiologica 


ones (for example, associations, bonds, habits) arc employed (Spence, 19512, 
P. 241). 


Already much attention has been given to the reasons for doubting 
the adequacy of S—R reinforcement theory, i.c. 
ing is a question of strengtl 
bonds by means of reward or 
more complicated; and as we 
come into proximity with 
contiguity theory, 


the notion that learn- 
ening or weakening stimulus-response 
punishment. The situation is obviously 
recognize this greater complexity, we 
what has been commonly termed. S—S 
which Spence (19512) has characterized thus: 
The cognition theorists 
thoroughly committed | th 
ing involves establishir 
ceptual processes. Cer 
stand on the matter, 
group as Lashley, 
terms of the form 
cessive perceptu 


» on the other hand, appear to be much morc 
an the S—R associationists| to the view that | 
ag relations (organizations) between sensory or per- 
tainly all of them have taken a definite and explicit 
Thus the Gestaltists, and such other psy chologists in this 
Adams, Zener, Lew in, and Tolman, conceive of learning in 
ation of cognitive patterns in which are organized the suc- 
al processes occurring in a behavior sequence (p. 243). 


What, more Precisely, does the SS contiguity theory imply and 
how does it articulate v ith or differ from present theory? ‘Fortunately, 
in 1947, R. S Woodworth published a paper which not only delineates 
the S—S contiguity position but also brings it into relationship with 
the concept of reinforcement. It was entitled, “Reinforcement and 
Perception,” and begins with the observation that 
incompatibility of [these] two factors, 
is that both of them are essential in 


It should be said that reinforce 


there is “no obvious 
and the thesis of this paper 
any process of learning? (p. 119). 
ment in the S—R bond sense docs 
not imply or logically demand any subjective concomitant whatever. 
But reinforcement, as p; 


avlov used the term, does lead to such a rclation- 


ship. This does not occur. of course, as long as one speaks of the 
g ne spe 
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«t items of behavior, i.e., of behavior in 
" But with the shift of emphasis from 
overt behavior to emotions, which has been previously traced in this 
a liaison emerges very naturally. To say, for 
‘raid or bopeful almost axiomatically im- 
and perception of objects and relation- 
anifestly has to do with ex- 
articular acts or move- 


(classical) conditioning of over 
terms of “conditioned reflex 


volume, the basis for 
example, that an organism is df 
plies a state of consciousness 


ships. And the learning thus implied m 


pectancies rather than with the execution of p 


ments, Says Woodworth, in similar vein: 
empathy aside, is that there is something 
present in the organism from the first trial on that is capable of reinforce- 
ment, and that this something belongs t0 the receptive and not to the efferent 
part of the organisurs total behavior. “Registration” fits the facts better than 
“perception,” for the human subject, at least, must perceive the sequence 

it may take many trials for the sequence to 
the appropriate motor or glandular response. 
be established before the conditioning 
cts the conditioning stimulus with 
first event in a regular sequence 


What I am struggling to say, all 


of events very quickly, while 
strike home enough to produce 
_ As to connections, several may 
is complete, but the prir one conne 
the meaningful character it acquires as the 
(pp. 121-122; italics added). 

is a quite explicit 
Woodworth, with the present analys \ 
than a simple SR paradigm for learning and is moreover suggesting 

à vs de "s vx cnpep pm * F 

that "learning" occurs in the first ot these two steps rather than in 
the second and that it consists basically of conditioning conceived as the 
rtain perceptions, expect 
terms all being more 


anticipation of present thinking. 


Here i 
. clearly, : 
„iş implying a two-step rather 


development Gres ations, anticipations, mean- 
ings, emotions—these or less equivalent (cf. 
Mowrer, 1960). 
For Woodworth, 
a derivative of the n 
He says: “In experim 


or response selection becomes 


habit formation selec 
arning just described. 


vore basic (sole?) form of le 
ents that offer alternatives and demand a choice, 
What has to be learned is a distinction between stimulus-objects and not 
between motor responses” (P 132): Woodworth does not indicate 
P hat a "perception" is, dynamically. or precisely how it can eventuate 
M one action rather than another. Bur the important thing is that he 
Makes conditioning the primary ) form of learning and de- 
"Ives response sclection therefrom. 

A somewhat similar approach wi 


' oh & Bitterman un 
Paper published in 1949 by Birch & Bitterman u Uu HER 
orcement and Learning: The Process of Sensory Integration." The up- 


shot of this study is that there arc tico forms of learning, conditioning 
S 5 [3 s «| S. m. " c 
tid temDundcenror at least. superficially different but that, 


(only? 


independently developed in a 
der the title, "Rein- 


which are 
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in ie final analysis, conditioning, conceived as “s ensory integration," 
is the more basic. And they cited Maier & Schneirla (1942 ) for the 
statement “that association by contiguity is more primitive than 
selective learning” (p. 304). Birch & Bitterman further anticipate the 
present analysis by noting that: 


Many of these changes in behavior apparently require the direct reorg: 
zation of afferent-efferent relationships. Since there is no reason to believe 
that proprioceptive stimuli cannot participate in the Integration process, we 
may assume that the specific proprioceptive consequences of the correct re- 
sponse may also come to acquire food properties (Grice, 1948b) (p. 306). 


ni- 


What these writers are here saving, if we understand them cor- 
rectly, is that so-called habit formation or response selection may be 
derived from the fact that response-correlated stimuli, if followed by 
primary reinforcement, take on the capacity to provide secondary 
reinforcement, which will subsequently cause this response to be 
selected, in preference to others, and not because an S—R bond has 
been strengthened but because the feedback from the R is “positive 
and facilitative. Inhibition or response rejection (as opposed to selec- 
tion) can be comparably explained. They say: “Evidently one can 
account for avoidance if one assumes that stimuli contiguously related 


to the onset of shock acquire the functional properties of shock" 
(p. 297). 

Up to this point the 
striking. However, something of 
Bitterman g0 on to suggest that 
the basis of sheer 
having 


parallelism with the present analysis is very 


a divergence develops as Birch & 
"sensory integration" can occur on 
contiguity of stimuli, | 
reinforcing properties 
decrement or 


without the second of these 
in the sense of implying cither drive 

àv: "The results of the sensory pre- 
i us to postulate a process of afferent 
integration), the essential condition for which 
of stimulation, and which takes place independently 
of need reduction lor presumably |" (p. 302) 

As long d to involve an EE 
necessary for 


increment, They 
conditioning experiments require 
modification (sensory 


is contiguity 


induction, 
as learning is assume 
question of the conditions 
tional basis: in order for le 
stimuli, Ss and S}, S 


—r sequence, the 
reinforcement has an opera- 
arning to result from the 


1 Must be capable from th 
And since the r's which 


e: 


pairing of the two 
[3 beginning of producing r- 
ime of reference, are most 
ionable are those of fear and hope and 
correlates of primary drive onset and 


. in the present fr; 
v condit 
since these are, respectively, the 
primary drive reduction, it 


ilv and most commonl 


Is obvious that the latter events are ul- 
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timately necessary for learning to occur. On the other hand, Birch & 
Bitterman seem willing to think of learning as also occurring in situa- 
and where, therefore, the sheer coincidence 


tions where there is no r 
of stimuli suffices to produce new "sensory integrations." 

To the present writer it is not at all clear what a "sensory integra- 
tion" is apart from a response of some sort. The response certainly 
need not be one involving overt, voluntary behavior; in fact, the whole 
force of our argument up to this point has been to show the error of 
i cither as the result of S—R bonds or of 
simple, direct conditioning. More commonly, the R or r which is 
learned as a result of stimulus pai ing is in the nature of an emotion. 
And it is even probable thar in “purely cognitive" learning there is 
a "response" of some sort within the central nervous system, 

ssumption can be illustrated by reference to 


The pertinence of this l : 
the phenomenon of “sensory conditioning or what is sometimes also 


oning.” Since t 
y supporting their 
: discussed in this book, 
presents the problem well, in these words: 


conceiving of such behavior 


his phenomenon is cited by Birch 
S—S contiguity theory and 
it should be given some 


known as “preconditi 
& Bitterman as cruciall 
since it is not elsewhere 
notice here, Osgood (1953 
had obtained evidence for “sensory con- 
presented simultaneously two 
ater made the CS for con- 
with shock serving as US, tests with the other 
been paired w ith shock, showed that it, too, had 
become capable of eliciting the new response. Certain controls were ap- 
plied: pretests with both stimuli showed that they did not initi lly elicit this 
reaction: animals treated identically, except for the original pairing of tone 
and light, gave no evidence of transfer. This was a highly significant ex- 
periment because it bore directly upon this fundamental issue of psychology: 
Is tbe modification tbat takes place n learning a central, cognitive event 
(S—S9 or MARE SOME behavior (S—R) be involved? The results of this ex- 
periment made it appear that the simple contiguity In experience of purely 
sensory events was sufficient for learning (p. 461)- 


) reported that he 
A bell and a light were 
of these stimuli wa 


Brogden (1939 
ditioning" in dogs. 
hundred times. When one 
ditioned forelimb flexion, 
stimulus, which had never 


iew a number of subsequent studies, some of 


Which confirm the Brogden findings and some of which do not. But the 
antly positive—and substantial enough to be taken 


been intimated (Chapter 7 Seron IN) a 


Osgood goes on to rev 


evidence is predomin 


seriously. As has already j 
comprehensive conception of the psychology of learning seems to re- 


quire that conditioning be thought of as applicable, not only to affective 
but also to so-called cognitive responses, in the 


processes (emotions). 
‘This hypothesis 
discussion, let us suppose that in the “pre- 


sense of images. again shows its cogency herc. 


In order to clarify the 
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conditioning” phase of the Brogden experiment, the bell Unt n 
instead of being presented simultancously, arc given in the bond HE ; 
light. Later, then, the light is paired with shock (w hich jaime id 
and leg flexion); and when the bell is then presented, it too produces 
fear and leg flexion. On the basis of evidence that will be reviewed else- 
where (Mowrer, 1960, Chapter 5), we may confidently assume that 
the light used in this experiment produces a light sensation (a “reaction 


in the dog’s head) which is conditionable in the form of a light image. 
Such a reaction, to be sure, is central or 
behavioral; 


“cognitive,” rather than overt, 
but emotions also are covert, nonbehavioral, vet we have 
not refused (at least not in recent decades) to 
theoretical system. And if we grant that sens: 
in the form of images, then the mystery ) 
solved: The bell, after pairing with the light, arouses an jage of the 
light; and after light itself has become capable of arousing fear and leg 
flexion, we would expect the latter re 
from light. itself (sensation) to the conditioned image thereof, as 
elicited by the bell. In other words, if, after the conditioning of fear and 
flexion to light, the bell merely “reminds” the subject of light, this 


alone may suffice to produce (or mediate—see Mowrer, 1960, C apter 
2) the defense reactions. 


The principal difficulty about 
sory” conditioning is that it is no 


admit them to our 
ations are conditionable 
of sensory conditioning is 


actions to generalize ( Chapter 12) 


postulating this type of “purely mele 
t at all clear when such conditioning 
occurs and when it does not. Not "very stimulus that precedes another 
Serves to remind us of it, even though the 
occurred many, many 


association has perhaps 
times.!? Something val 
tion" is undoubtedly. 


ucly known as "atten- 
in so 


operational definition the 
cannot, therefore, em 


me way relevant here 
reof (but see Mowrei 
ploy it systematically, 

Hence, in the present. writer's judgment, it is misleading and 
unrealistic to talk about “S—s contiguity” learning. What we have 
instead, it seems, is aly ays S—S—_R or, more exactly, S—S—r learning; 
and for this the term “conditioning” seems generally adequate and 
preferable to such expressions as “perceptual reorganization? “sensory 
integration,” “afferent modification," and the likes although, in many 
other respects, the kind of thinking developed both by Woodworth 
and by Birch & Bitterman is entirely congruent with our own and 


s but as ver we have no 
"^ 1960, Chapter 5) and 


" This is perhaps w hy 
tioning, is erratic: sometin 
variables which we have 
control. 


the phenomenon of sensory conditioning, 
ts vou get it and sometime T 
not ver clearly 
For a recent literature review, s 


or precondi- 
es you don't, depending upon 


identified and are not vet well able to 
see Seidel (i959), 
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importantly supplements and supports it (see also Birch & Bitterman, 


1951). 


VIII. SCR Contiguity Theory 


"The group of theories which are to be discussed in this section have 
alwavs been hard for the present writer to "understand." Perhaps it is 
the fact that they are, as one may say, “antitheoretical” theories that 
creates this difficulty. Skinner has been very frank to declare his aver- 
sion to formal theorizing; Guthrie has said that his views “really do 
not constitute a system “or complete theory, just a prejudice in the 
direction to be taken” (personal communication—cf. Hilgard, 1956, 
pp. 78-80); and Estes is concerned. with essentially statistical rather 
than. psychological variables. Nevertheless, the writings of these three 
men have attracted interest, inspired research, and stimulated others 
very much as one might expect bona fide theories to do. We shall 
therefore have to give them careful attention, hopefully with the view 
to finding ways in which they articulate with present thinking. 

Let us. begin with the following summary by Spence of Guthrie's 


position: 


governing the development of association. between 
n ents is association by contiguity. This prin- 
1 that is acting at the time of a response will, 
sponse" (Guthrie, 1942, p. 23). The es- 
sential and sufficient condition for learning, according to this principle, is 
simultaneity of stimulus pattern and respon: Motivation, in the form of a 
drive state, such as hunger, and the US serve me sly to assure that the response 
to be vondicioneu will occur. Their function is that of a "forcer" of the re- 
sponse rather than that of a “reinforcer” of a connection (Spence, 1951b, 


P- 703). 


The main principle 
stimulus and response movem 
ciple states, “A stimulus patteri 
if it recurs, tend to produce that r 


and in those of some of his students this 
some supplementary assumptions) can 
ance of accounting for a wide range of 


In Guthrie's own hands 
formulation (together with 


be made to give the appear ; 
behavioral phenomena. But it 1s hardly a theory of learning in any 


very conventional sense of that term. Ordinarily one thinks of be- 
havioral change as the hallmark of this process, w hereas Guthrie's 
gically related to the prediction of behavioral 
This can be illustrated by the kind of exper- 
imental evidence which Voeks (1948) has adduced in support of 
Guthric's position. A corollary of the principle already quoted is that 
the response last made to a given situation or stimulus pattern will be 


principle is really more lo 
constancy or continuity. 
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the response most probably made when that eranan ‘a vix ar nel 
tern recurs. And Vocks analvzes data reported by l ace ies hia 
and herself to show that what she calls the “principle a i dii 
(lastness of response) is highly prognostic. She says, for cxample: 


Of those occasions when the postreme response had been) uc ed i 
Sponse, 73.0 per cent of the time it was made on the next trial. eei i 
casions when the postreme response had been the w ang Eos cab pte 
63-9 per cent of the time was it made on the next trial.” vp : Se dicens 
that if postremity tends more tow ard preserving one SEa Fes} ncs oie 
than another, it tends more toward preserving the correct Sas aa E m 
than the incorrect... . Thus it is found that cont ry to ] SECURUR jun 
clusions, his data indicate that postremity may be a significantly accura 
basis for prediction and is not negative in its effect (p. 497). 


ion i 7 redict? It is probably 
But the question is, What does one wish to predict? It is probably 


pee x ES S tbe 
true that the statistically safest prediction that one can make abou 
havior is that “next time" it will be the san 


: i > chances are it will 
other words, a given response tendency exists, the chances are it 
per. 


ist." But while this premise may have a 6454 -85'« patus 
(Voeks, 1948), it is Still 100% useless as far as giving us any way uF 
predicting or producing change." In reply to criticisms of his theory 
advanced by O'Connor (1946), Guthrie (1946a) has himself said: 

I am interested in the | 
effect psychologists are 
terested in the 


ae as it was last time. If, in 


aws of moven 
interested in the 
prediction of such p 


lents as determined by stimuli. The 
end results of action; . . . | am in- 
articular movement series (p. 289). 

From this one might infer 
predicting behavior 
to say: 


that Guthrie's theory is useful only for 
constancy, not change. But this writer goes on 
“My own interest has been in the question: under what circum- 
stances do responses to stimuli change and what constitutes. such 
change." The Postremity principle, at least as interpreted by Voeks. 
would seem to require simply that living organisms alw: 
they did “last time,” thus precluding the 
as the environment itself may change, 


vs do what 
yl ro except 
possibility of change E 
s : in which 
Yet there is a sense in whic 


“Tn weather forecasting 


+ it is well know n that 
same” will be correct well 


over 50% of the 
"Hilgard (1956) has voiced the 


cannot predict when the first CR 
the principle (ar least, so far 


fedi m : c 
a prediction of "more of th 
timc. A 
Same objection: “The postremity princip. 


i us violates 
Appears, because its appearance always viola 


as the recorded responses are concerned) (qe 74)+ 
: d b s iio 
And Miller (19583). has similiarly said: ^] have been unable to see... why 
| Guthries theory | should not 


predict thar a thoroughly learned response to 5 
r extinguish. | There is also the problem of why 
the stimulus-change produced by the o painful stimulus does not protect 
preceding responses from re (p 261). 


momentary stimulus will neve 


"set of a 
troactive inhibition 
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Guthrie’s thinking seems to provide for genuine learning, i.e., of at 
least a limited kind. In the passage already quoted from Spence, it is 
pointed out or at least implied that if a particular stimulus occurs and 
ular response, other stimuli will have a tendency to 


“forces” a partic 1 [ y to 
produce the response.” But this is nothing but the “redintegrative 
theory of conditioning which Hollingworth (1928a, 1928b; see also 
Thorndike 1913, 1932) has advanced long since.'* Therefore, at this 
point Guthrie’s theory ceases to be an S—R contiguity theory and 
‘stance of S—S—R contiguity, or conditioning. 
arly primitive, unclaborated version thereof, 
‘and difficulties so extensively explored in 


becomes simply an inst 
It is, moreover, a particul 
subject to all the objections 
carlier chapters. ? 

Yet on more than one occasion other psychologists have reported 
seeing a similarity between Guthrie's position and the onc delincated 
in this book. Perhaps the key to this impression 1s the emphasis in both 


upon response-produced stimuli. Hilgard (1956) gives a good summary 


of Guthric’s position in this connection as follows: 

One of the standard experiments in the literature of conditioning is that 
showing the importance of the time interval between the conditioned stimu- 
lus and unconditioned response. The empirical results suggest a gradient, 
with a most favorable interval and less favorable intervals on either side of 
this optimal interval [see Chapter 0]... à x 

Chre is able to hold out for strict simultaneity of cue and response in 
the face of ihesa data by proposing that the true cue being conditioned is not 
the smuls as measured An external stimulus may give rise to movements 


of the organism. These movements in turn produce stimuli. When associa- 
mae tween stimuli and responses separated in time, 


movements fill in the gap. The truc association 


tions appear to be made be 
it is because these intervening 


is between simultaneous events. ; j 
There is a strong preference for movement-produced stimuli as the true 


conditioners in Guthrie's system. They permit the integration of habits within 
à Wide range of environmental change in stimulation, because these stimuli 

y the organism. It appears that some of this preference 
asis of Watson (1907) on kinesthesis as the basis 
abit, a position no longer tenable. Such covert 
li provide ever-present explanations for conduct 
rom external stimulus-response relationships 


are carried around by 
dates from the carly emph 
of control of the maze h 
movement-produced stimu | 
Which cannot be inferred f 
(p. 54). 

Despite Hilgard's 
stimuli" which are the 
as the response occurs are the future cues for the 


observation that it is "movement-produced 
"true conditioners in Guthrie's svstem," the 


"'"Dhe stimuli present 


response" (Vocks, 1950)- ii i tubes God hi T 
wa the revised. edition of his book Guthrie 11952 himself uses the term 


"redintegrates? and hails Pierre Janet as its principal carly exponent. 
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fact is that the resulting similarity to our own position remains very 
faint. When Hilgard speaks of intervening movements "which fill in 
the gap," it is clear that the mechanism here posited is the old, 
familiar, and relatively barren one of reflex chaining. This is a notion 
which was popular in the heyday of Behaviorism but which finds few 
contemporary advocates (cf. Keller & Schoenfeld, 1950). The present 
theory stresses movement-produced stimuli for quite a different pur- 
pose. 

Reverting now to Guthric's suggestion that a response is "condi- 
tioned" to the total situation or stimulus "population" present at the time 
of its occurrence, we find that two interesting implications flow from 
this. One is that it leads directly to Estes’ "statistical" theory of learn- 
ing, which will be considered shortly. The other is more closely 
concerned with present theory. In Chapter 7 evidence is cited for believ- 
ing that if a previously rewarded response, when it recurs, produces 
all or most of the stimuli which it has previously produced when thc 
response has been followed by reward, that response, owing to the 
secondary-reinforcing or incentive power of these stimuli, will have 
maximal “habit strength” and that habit strength will be progressively 
less as the number of response-produced stimuli is reduced. Thus, in 
both Guthrie's system and in ours, the likelihood of 
forced response’s occurring is a function of the "similarity" of the 
"situation" in which the response was reinforced and the now prevail- 
ing situation. But therc is a silent difference: For Guthrie the "situation" 
that is important is made up of that pattern or population of stimuli 
which is said to initiate or “cause” 


a previously rein- 


the response, whereas in revised 


two-factor theory the “situation” that counts is that pattern or popula- 


tion of stimuli which is the consequence of the response as it occurs 
(or is symbolically "contemplated"; see. Mowrer, 1960, Chapter 6). 
Whether a response will or will not occur thus de 
upon the similarity between present and past objective situations, 
but rather upon whether it produces 


response-related stimuli which 
have "cathexis" relevant to the organism's current needs. Thus, “habit 


strength” is made to depend crucially upon the feedback from a re- 


pends, not especially 


" Professor Guthrie h 


as himself stated: “The differences between our positions: 
are very profound” 


(personal communication). 

“Guthrie's special interest in mov ement-cued behavior chaining arises for the 
reason that delayed reinforcement constitutes a very difficult 
system. Holding that it is alw ( 


is “learned.” it is hard to expl 
act a: a distance 
reinforcement is h 


problem in his 
ays the Jast response made to a given situation that 
ain how a reward (or punishment) can, so to say. 
‘see O'Connor, 1946; Guthrie. 19462). The way the 


delay of 
andled in the present system is described in Ch 


apter to. 
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sponse, rather than upon familiarity or similarity of the initial situation. 


Hilgard quotes Guthrie thus: 


“Drinking or smoking after years of practice are action systems which can 


be started by thousands of reminders. . - l had once a caller to whom I 
was explaining that the apple | had just finished was a splendid device for 
avoiding a smoke. The caller pointed out that | was at that moment smoking. 
Fhe habit of lighting a cigarette was so attached to the finish of eating that 


smoking had started automatically" (p. 56)- 


laptive capacity of living organ- 
The reality and high psychological 
special consideration later in 
pothesis which is currently 


; Does this incident represent the 
isms at its best, or quite otherwise? 
relevance of cues will be given 
this volume (Chapter 12); but the hy 
regnant is that it is the expectations whicl 
arouse, rather than stimuli that have previously produced the response, 
evant here. As Rogers (1956) has observed in 
) book on clinical psychology, 
important than the past: ". . . 
not backward—which 


H response-produced stimuli 


that are most highly rel 
of George Kelly’s ( 1956 
ture is more 
looks forward, 
not reactive” (p. 357)- 


à recent review 
for living organisms the f’ 
a theoretical system which 
sees behavior as anticipatory, 


IX. The Views of Estes and Skinner 


itistical theory of learning is a direct and natural 


ric's formulations. 
potheses as essentially mathematical eqt 
verbal analyses” (p. 107)- With Guthrie, s 
occurrence of a response, R,, all new elements (.e.. clements not al- 
ready conditioned. to Ra) in the momentarily effective sample of 
onditioned to Ri” (p. 97). And the likclihood of 
\ elicited is determined by the extent to which 
j different stimulus elements (ef. 
es converts this hypothesis 


Estes" (1950) ste 


EUER ES meres regards his i 
outgrowth of Guth In fact, Estes regards his hy 


tivalents of "Guthrie's (1946b) 
Estes assumes that "on cach 


clements, s, become ¢ 
Ris being subsequentl 
a later situation contains the same or ditt 
Chapter 7, p. 234): The way in which Est i 
into mathematical terms is ingeniou and his attempt thus to overcome 
some of the rigidity. and oversimplification of traditional stimulus- 
Ses principal advantages" (p. 106) 


Can even the most clegant 


response theory without abandoning 1t5 


is very much worth while. The question 15: f 
-ansformations redeem stimulus-response theory from 


Following Guthrie (and Skinner), the par- 
se theory which Fstes accepts as the 


of mathematical tr 
its basic disadvantages? 
f stimulus-respt 
his equations 
this writer, 


ticular brand o 
logical basis for 
dubious form. For 


is behaviorism in its most extreme and 
the. independent variable (cause) is 
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the “environment” of the organism and the dependent variable (effect) 
is the organism's “behavior.” And the “connection” is direct and. un- 
complicated by intervening variables. Thus, S—R is the basic par: digm, 
without so much as an allusion to the more intricate S—r: s—R formula 
which previous chapters have shown to be minimally es sential." 
Prototypical of the inherent weakness of such an approach is the di- 
lemma suggested by the following quotation: 


In the familiar buzz-shock conditioning experiment, for example, S. would 
represent the population of stimulus elements emanating from the sound 
source and R would include all movements of a limb meeting certain speci- 
fications of direction and amplitude; typically, the R to be conditioned i 
flexion response which may be evoked on cach training trial by administra- 
tion of an electric shock (Estes, 1950, p. 98). 


Here it is assumed that the *R to be conditioned" is the same as that 
produced “on each training trial” by the unconditioned stimulus, or 
shock. But we know (Chapters 2 and 3) that the “conditioned” re- 
sponse is often radically different from the unconditioned 
and the only satisfactory way of accounting for this f. 
postulation of fear as an intervening variable. Although it is a good 
thing to have psychologists (like Hull and Estes) who have the ability 
and the motivation to translate “purely verbal" hypotheses into 
mathematical form, the resulting derivations and inferences will be 


no better than the assumptions that have gone into them, 
is not aware of any 


response; 
act involves the 


‘The writer 
instance where the formalization of a theory of 
learning in mathematical terms has importantly advanced 
standing and know ledge (scc Chapter 7, Section III). 


such formalizations have merely produced 


our under- 
Morc commonly, 
à deceptive appearance of 
nmature stage of develop- 
ment. The writer recalls having once heard Clark Hull 
conversation with Max Wiertheimer, in the 
latter had maint 


a finality which has fixated thought at an ir 


recount a 
course of which the 
ained, to Hull's evident consternation, that the formal- 
ization of psychological theory was then (in the 


The fact is that all Hull's equations did not keep his theoretical system 
from being inadequate, nor did the absence thereof prevent T'olman's 
speculations from containing 
Chapter 9). 


A few vears ago Spence wrote that, 
is developed, the essential similarity of its formal structure to that of Hull's 
theory will become more and more apparent" (Spence, 195 1b, p- 706). Since 
it remained for two admitted "fifth columnists” (MacCorquodale & Mechl, 


ples of the theory in action, see Brody 
& Frankman (1957), and Anderson & Grant (19 


1930S) "premature." 


à more generous segment of truth (see 


in his opinion, “as Tolman's theory 


"For some recent. exam 
Burke, Arkinson, 


(1957), Estes, 


57). 
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“formalize” Tolman’s theory, such an outcome would not 
it appears that Spence would have pre- 
an with Hull in the above quota- 


1933. p- 63) to 
be too surprising. More surprising, 
dicted better if he had interchanged Tolm: 
tion (Seward, 1956b, p. 111). 

With this admission of skepticism about the utility (to date) of 


mathematical formalizations of learning theory, we turn to an even 


profounder form of doubt, namely B. ^. Skinner's (1950) questioning 
of the fruitfulness of theory itself. This. as one might guess, is a 
qualified kind of skepticism, it being admitted that “it would be 
foolhardy to deny. the achievements of theories . . . in the history 
of science" (p. 194). Skinner's objections, such as they arc, can be said 
to be these: (a) "Much useless experimentation results from theories, 
and much energy and skill are absorbed by them” (p. 194). When 
Hull's theories are criticized, they are sometimes obliquely defended 
by the rejoinder: “But look at all the research they have stimulated!” 
The question is whether this research has any value, other than as its 
confirms or refutes the theory. Suppose, as now seems probable, that 
the net effect of the research which Hull inspired is to show the basic 
unsoundness of his system. Does this mean that both the theorizing 
and the research were a waste of “energy and skill?" Skinner is in- 
clined to think so. And (b) “It might be argued that the principal 
function of learning theory to date has been, not to suggest appropriate 
research, but to create a false sense of security, an unwarranted satisfa 


tion auch: rhe aratris quo". (p: 194)—and, one might add, a tendenc) 
ave found their sense of security in different theories 
ment. Certainly Skinner's own professional 
and its scientific and practical im- 
respectful consideration of the 


among those who h 

to dispute rather than experi nt 
carcer, in terms of its productivity 
as tO command 
sses and exemplifies. 

is relatively simple. We need to do 
re observations of a purely em- 


pact, has been such 
alternative which he prop 
His programmatic “platform 


many more experiments, make many mo em 
pirical kind obtain more "data showing orderly changes characteristic 
al kind, obtz £ à 8 


of the learning process" (p. 215). Theories are, admittedly, “fun.” but, 
say's Skinner, we are not ' for them. And we are especially far re- 
D SOK ` * TE x " 

moved from the time when we can fruitfully cast our theory into 


mathematical form: 


‘ready 


little effective use of empirical, let alone ra- 
nt curves could have been fairly closely 
fitted. But the most elementary pr liminary research shows that there are 
many relevant variables, and until their importance has been experimentally 
determined, any equation that allows for them will have so many arbitrary 
constants that a good fit will be a matter of course and a cause for very 


little satisfaction (p. 216)- 


At the moment we make 
tional, equations. A few of the pr 
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Skinner even distrusts so rudimentary a statistical device as the use 
of averages: Let us, for the moment, stick to the detailed observation 
of individual subjects, he says. And, above all, let us avoid that selec 
tivity of observation and attention that permits us to "prove" di- 
ametrically Opposite assumptions, depending on our vantage point. 

Why, then, should Skinner be discussed in the context of the highly 
mathematical work of Fstes and the highly speculative (if not “the- 
oretical”) formulations of Guthrie? Like the two investigators just 
cited, Skinner is, in the first place, a radical Behaviorist, a thorough- 
going empiricist, and essentially an operationist. It would hardly be 
fair, in view of his explicit repudiation of theory, to call him an S—R 
contiguity theorist; but he is at least one SD 


association," in terms of 
those who have been attracted or influenced by his work. Nor would 
we have him be other than what he is. 
techniques, methods of qualification, and ingenious "schedules of rein- 
forcement" (Skinner, 1953b; Ferster & Skinner, 1958), Skinner has 
shown himself extraordinarily resourceful and creative. Would. that 
there were more of his ilk. But this is not to say that his “dust-bowl 
empiricism” brand of science is alone sufficient. As the very title of 
this book implies, the present writer believes, with others, that some 
sort of balance between theory and observation is desirable; and it is 
not surprising that Skinnerians feel that they have to take an extreme 
position if they are to counteract the dead weight of the all-too- 
numerous "pure thinkers.” 

The present writer regrets that he is not able to make a more ex- 
tended comparison of Skinner's “system” with the one under review 
in this book. This limitation comes in part from the somewhat enigmatic 


nature of Skinners work, Verplanck (1954) has succinctly 
summarized thus: 


In terms of behavior-control 


which 


[Here] we are concerne 
a systematist whose 
some of whose 
(p. 268). 


d with a theorist who now 
system is still deve 
most import 


espouses no theory, 
loping, and a constructive thinker 
ant contributions have been those of a critic 


f In his review of current research and theory in the field of learning, 
Kendler (1959) has been particularly thorough 
praising the work of Skinner and his followers 


deal of space to Estes "statistical le 


in covering and ap- 
(he also gives a great 
arning theory”); and here we read: 
Skinner is an enigma. His attitudes, 


à "s and those of many of his partisans 
toward facts, theories, statistics ind science itself are incongruous. But in 
spite of this his work, as well as that of his rapidly growing legions of ad- 
herents, is fascinating. and creative, and has much of permanent value 


(p. 59) |cf. Mowrer, 1960, Chapters 8 and oJ. 
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This "option-plav" (or “playing-by-ear”) type of research generates a 
mass of data that is most difficult to analyze 'stematically. Each experi- 
mental article contains many more variables than subjects. The knowledge 
obtained, as well as the interpretation offered, is that some variables, in 
conjunction with other variables, result in changes in behavior (p. 61). 

Where does the Skinnerian formulation go from here? At present the 
Skinnerians have been able to generate an enthusiasm and conviction among 
themselves embling a quaint mixture of that found in a revolutionary 
partv, a revival mecting, and a homecoming football gathering. Can this 
enthusiasm and conviction maintain itself on such a skimpy diet of theorctical 


notions? This writer thinks not . . . (p- 65). 


) evaluation of Skinner's strategy reads, in part, 


And Hilgard’s ( 1956 
as follows: 
ach to psychology has made it difficult for 
hers, and, on principle, he rejects their 
he theories of others—insofar as he has 
paid any attention to their claims—has been chiefly that of a trenchant critic. 
He has felt no responsibility for the task of inter-investigator coordination. 
In his book, Science and buman behavior (1953), Written as a textbook, he 
used no literature citations, and he mentions by name, among writers with 
some place in learning theory, only Thorndike, Pavlov, and Freud (p. 116). 
(1950). while remaining within Skinner's 
“system” (i.c. his conception of what is psychological science), have 
been somewhat more eclectic. And Schoenfeld (1950), writing alone, 
has a bibliography of 42 items following a comparatively short paper. 
Here, in good Skinnerian tradition, he sets out to “recast the anxiety- 
reduedon hypothesis" as it pertains to "av oidance conditioning" (p. 77). 
with results that scem to retard rather than advance the general level 
of our understanding in this area (sce Mowrer & Keehn, 1958). But in 
one important respect. Schoenfeld's discussion breaks new ground: it 


explicitly calls attention t 


stimuli in the control of beh: 
asis upon this phenomenon in present theory. Schoenfeld 


Skinner's "fresh start" appro 
him to use the data collected by ot 
concepts. His role in reference to t 


Keller & Schoenfeld 


) the probable role of response-correlated 
avior, in a way almost exactly parallel to 


the emph 
says: 


In addition to the termination of ave sive stimulus compounds [Schoen- 
feld’s circumlocution for fear], it is possible that another source of rein- 
forcement comes into play. The proprioceptive stimuli produced by the 
avoidance response may. because they are correlated w ith the termination of 
noxious stimuli, become secondary: positive reinforcers and hence strengthen 
the rendcnev to makethe TFOSpanse Nw hich generates them (p. 88). à 


And again: 

avoidance response terminates stimulus compounds 
and tactile stimuli are important components. Re- 
cited to support this formulation, are interpreted 


It is argued that the 
in which proprioceptive 
cent experimental data, 
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indicating that proprioceptive stimuli generated by the organism's own 
Movements can act as secondary negative or positive reinforcers and thus 
control the movements producing them (p. 97 


Although we would not at all agree with Schoenfeld in his “sanitized” 
(and essentially circular) definitions of “secondary negative and posi- 
tive reinforcers,” the fact that he sees and stresses the implications of 
their being conditioned to the “proprioceptive stimuli generated by 
the organisurs own movements” constitutes one more important indica- 
tion of the extent to which investigators of the most diverse conceptual 
backgrounds are today arriving ata basically similar position. All roads 
do in truth, it seems, "lead to Rome." 


X. Konorski's Version of Two-Factor Theory 


An unsympathetic reader of this chapter might be tempted to ob- 
serve that thc theory here under review has drawn much of its support 
thus far from the "soft-" rather than from the "hard-headed" wing of 
psychological science. Certainly we have not been excessively im- 
pressed by the pretensions of the ultra-objectivists in the domain of 
learning theory; and we have not hesitated to listen respectfully to 
those who admit to employing both concepts and data of a frat 
subjective kind. But this is not to say th 
from some extremely sober 
earlier chapter the intern 
Konorski, has alre 
two-factor theory 


ankly 
y that support does not also accruc 
and empirical kinds of inquiry. In an 
ationally known physiologist, Professor J. 
adv been cited as an advocate of an carlier version of 
t sand it will now be our purpose, within the remain- 
ing limits of space in this chapter, to show that, quite independently, 
his laboratory researches in recent vears have moved along lines very 
similar to those we are exploring here. This parallelism becomes espe- 


cially evident in a paper published in 1950, Which reads in part as 
follows: i 


In 1928 we succeeded in separating out a ne 
different from the Pavlovian conditione 
ditioned reflex of the second type” (Miller & Konorski, 1928). On the basis 
of our experimental work concerning this type of reflex (vide Konorski, 


1948, Chapter xi) the principles of its elaboration can be generally formu- 
lated as follows: À 


If we subject to conditioning Procedure of the first type (i.e 
by an unconditioned stimulus) a compound of stimuli cons 
exteroceptive and a proprioceptive stimulus, in which the proprioceptive 
stimulus. constitutes an indispensable complement to the conditioned com- 
pound, then the exteroceptive stimulus begins to e 


w form of conditioned reflex, 
d reflex, which we called the "con- 


reinforce 
ting of an 


voke either tbe movement 
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generating the proprioceptive stimulus or the movement antagonistic to it, 
which depends (1) on zz hether the conditioned reflex, first type, to the com- 
pound is excitatory or inbibitory, and (2) on whether tbe reinforcing stimulus 
is positive or negative. By positive unconditioned stimuli we denote such 
agents as food, an individual of other sex, etc., and by negative unconditioned 
stimuli we denote such agents as the introduction of acid into the animals 
mouth, the electric shock, etc. 

Thus we obtain four varieties of conditioned reflexes, second type. They 
are as follows (s denotes an exteroceptive stimulus, s, proprioceptive stimulus, 
r corresponding movement, ~ r antagonistic movement, > evokes): 

(1) If the compound $+ $r is reinforced by a positive unconditioned 
stimulus, stimulus s alone not being reinforced, then stimulus s begins to 
evoke the movement r (s 7). 

(2) If the compound s + s, is not reinforced by a positive unconditioned 
stimulus, while stimulus s alone is reinforced, then stimulus s begins to evoke 


the movement antagonistic to T Co T). d 
(3) If the compound s + $, is not reinforced by a negative unconditioned 
stimulus, while stimulus s alone is reinforced, then stimulus s begins to evoke 


the movement r (s T). ; Md z 
(4) If the compound s + Sr 1$ reinforced by a negative unconditioned 


stimulus, while stimulus $ alone is not reinforced, then stimulus s begins 
to evoke the movement antagonistic to r (5 ~ r) (p. 418). 

Near the end of Chapter 7, an experiment by Loucks has been de- 
scribed which nicely exemplifies the first of the four learning situations 
here listed by Konorski. In that experiment, it will be recalled, a 
buzzer and proprioceptive stimuli produced by cortical excitation of 
leg flexion, in the dog, were followed by food; and soon the subject 
the leg whenever the experimenter sounded the buzzer. 
corresponds, obviously and exactly, to the circum- 


was flexing 
This procedure 


stances described in (1) above. 
Now let us suppose that in the Loucks experiment buzzer and corti- 


cally induced leg flexion had been followed, not by food, but by a 
1 t of some sort. Thereafter, when the dog heard the buzzer, 


punishmen : hii. 
other "danger signals" (i.c, proprioceptive 


if it started to lift its leg. 


stimuli) would be introduced I l ; 
reverse movement of leg extension, because this would eliminate the 


stimuli associated with leg flexion. This procedure 
ie one described in (4) above—and is reminis- 
ago reported by Pavlov (1927). 

(3)? How are we to interpret or identifv 
ill be clearer if we describe it a little 


; and the dog might very w ell make the 


proprioceptive 
would correspond to tl 
cent of an cffect long 


But what of procedure 


it? Perhaps its implications W 


differently. Suppose We say: 
owed by a negative unconditioned stimulus (such 


If s alone is follo ! 
as shock or some other disagreeable experience) and if the compound 
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s+ s, is not followed by the negative unconditioned Stimulus, then 
d J h 
s begins to evoke the movement r (s > 7). 


What the dog discovers here is that by lifting his leg and producing 
S, he can, so to say, “nullify” the s alone and prevent the negative 
unconditioned stimulus from occurring. In other words, the dog finds 
that s is a danger signal but that s +s, is not. Thus, the fear aroused 
by s is reduced by the occurrence of s, Hence the dog will be re- 
warded, by secondary reinforcement (type-1), for making the leg 
flexion response. It will therefore tend to occur. promptly and specifi- 
cally, whenever s is presented (cf. Mowrer & Keehn, 1958; Sidman, 
1953; and Chapter 12). 

Thus Konorski’s situations (1), (4), and (3) are readily understanda- 
ble in terms of principles already established in our preceding discus- 
sions. What, now, 
helpful. 


of situation (2)? Here again a restatement may be 


(2) If a stimulus s alone is reinforced by a pos 
stimulus (e.g., food) and if the 
when the subject gets the 
response r and thus nullify 
against the occurrence 
antagonistic to r. 


ve unconditioned 
compound s + s, not reinforced, 
stimulus s it will be careful not to make 
the reinforcement, and one way to guard 
of r is for the subject to make the movement 


With this reformulation Konorski's situation (2) becomes quite as 
intelligible as the other three. 

It is apparent that 
simpler learning 
of them, wh 


discrimination (see Chapter 135): as opposed to 
. is involved in all four of the above situations, In all 
at the animal has to le 
happens if s alone occurs and wh 
tion (1), s will be followed by 
(food) only D s 
incentive for m 


arn is the difference between what 
at happens if s + s, occurs. In situa- 
primary decremental reinforcement 
accompanies it; therefore the animal has a strong 
aking r. In situation ( 
incremental reinforcement (shock 
fore, again, the subject has a stro 
tion (2), s will vot be followe 
ment (food) if it is accomp 
for seeing to it that r 
the reciprocal of r). 


3). 5 will be followed by primary 
) if s, does not accompany it; there- 
ng incentive for making r. In situa- 
d by primary decremental reinforce- 
anicd by Sr, SO that the subject has reason 
does not occur (and to this end may Instigate 


And in situation (4), s will be followed by 
primary. incremental reinforcement (shock) if it is accompanied by 
s, so that the subject again has reason for mal 


king sure that r does not 
occur (and to this end may 


instigate the reciprocal of r). In two of 


these situations, s, is. in other words, a positive cuc in the sense that it 
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ensures decremental reinforcement (1) or prevents incremental rein- 
while in the other two situations, s, is a negative cue 


forcement. (3); 
emental reinforcement (2) or produces incre- 


in that it prevents deer 
mental reinforcement (4)- 

From an intuitive, common-sense point of view, we can see why, 
in situations (1) and (3), r is positively motivated and why, in situa- 
toins (2) and (4), r is negatively’ motivated (and its reciprocal posi- 
tively motivated). But how, from the standpoint of revised two-factor 
theory, do these facts look? At once we sce that in situation (1) the 
stimuli which are correlated with the occurrence of r take on sec- 
ondary-reinforcement properties (of the hope variety), thus providing 
the conditions for “voluntary” (“habitual”) performance of this re- 
sponse. And we see that in situation (3) the stimuli which are corre- 
lated with the occurrence of r likewise take on secondary-reinforce- 
ment properties (but of the relief variety), again providing the condi- 
tions for the performance of this responsc. In situation (2), the stimuli 
which are corrclated with the occurrence of r acquire secondary 
morivano properties (of the disappointment variety) and provide 
inhibition of this (performance of the antagonistic) 


conditions for the 
the stimuli which are correlated with 


response. And in situation (4), \ 
of r acquire secondary motivational properties (of 


and provide conditions for the inhibition of this 
of the antagonistic) response. Thus, we have exem- 
f secondary reinforcement, two decremental 
Table 7-1, with intrinsic stimula- 


the occurrence 
the fear variety) 
(and performance 
plified all four forms o a 
and two incremental, as shown 1n 
tion. 

These four ex} 
four verbal injunctions to t 


serimental situations may be very simply presented in 
he subject: 


(1) “Now (when the buzzer sounds) if you lift your foreleg, you 


will get food’ —hope. ' 
(2) "Now if vou lift vour foreleg, vou will vot get food" —dis- 
appointment. : y : 
(3) "Now if vou lift vour foreleg, you will vot get shocked"— 
relief. 
(4) “Now if you 
will get shocked” — fe 


lift vour foreleg when the buzzer sounds, you 
ar. 
in so-called discrimination situations (Chapter 


imenter present onc of two or more possible 
subject to follow one rather than another 


We are accustomed. 
12), to having the exper 
ause the 


cues which will ¢ j i 
course of action. In all four of the situations here described, the 
except that the roles of command and obedience 


situation is similar, 
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are, in a sense, reversed: Here the subject causes the experimenter 4 
follow one or another course of action (present food, not present ison 
not present shock, present shock) according to à cuc which m je » 
ject gives to him (and to himself) in the form of a rama ud 
sponse. As in trial-and-error behavior generally, the subject B dm 
sponsible" for what happens, in the sense that, by responding or nigy 
responding ina particular way, he can ensure favorable consequences 
(reward) or incur unfavorable consequences (punishment). In one 
wav, the experimenter is, of course, controlling the behavior of x 
dog; but in another way the dog is controlling the behavior of thc 
experimenter. e 

What, then, is the role of the buzzer? It seems to serve mainly as 
an alerting or readying signal. It says, in effect, “Now is the time 
für. Le Te as a Constant in all four conditions, with the meaning 
that gets attached to the leg-lifting varying systematically. In situa- 
tions (1) and (3) hope and relief, respectivel 
the proprioceptive stimuli produce: 
tate it, whereas in situations (2) 
respectively, become attache 
sponse that 


become attached to 
d by this response and thus facili- 
and (4) disappointment and fear, 


d to these stimuli and inhibit the rc- 
produces them or instigate 


The four situations discussed abo 
Dr. Konorski and his 


them experimentally, 


an antagonistic response. 
ve arc not hypothetical ones: 
associates (and others) have reproduced all of 


And they illustrate, almost as if made to order, 
our hypothesis that behavior is controlled (or 


motivated) by two "positive" and two 
hope and relief, on the onc 

the other. When the stimuli 
response-produced, 
And, in the 


s in MeClelland’s terms, 
"negative" affective states— 
hand, and fear and disappointment, on 

Which mediate these four states are 
the results reported by Konorski are obtained. 
event that these stimuli are xot response-produced, what 
then? Already we know that the result is so-called approach or avoid- 


ance behavior, in the Spatial sense. In the next chapter we shall examine 
hese phenomena more c 


a 


arefully, 


XI. Further Indication of the Role of 
Proprioception in Habit 


As an illustration of the kind o 


f research which is now issuing from 
konorski's laboratory, 


let us examine a paper recently published by 
Stepien & Stepien (1958). report is compact and clearly 


ntial parts in the. authors’ own 


Because the 
written, we shall reproduce its esse 
words: 
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Experiments were performed on 9 dogs. In all the animals instrumental 
(type I) alimentary conditioned reflexes were established. The dogs were 
trained to lift the right foreleg to various acoustic stimuli (bell, metronome, 
buzzer, tone, etc.), and put it on the food-tray in front of the animal. [In 
other words, a discriminant operant (see Chapter 2, Section IX) was estab- 
a habit which was under stimulus control: the leg movement 
" only when the cue stimulus, or SP, was present.] The per- 
formance of this movement in response to the conditioned stimulus was re- 
inforced by food. Each daily experiment consisted of 8-9 trials. Usually, 
after several davs, the dog learned to perform the required movement in 


“conditioned stimuli. After 2-3 months conditioned reflexes 


lished, i 
would “wor 


response to the 


were firmly established. 
In some of our dogs a number of additional tests were performed in other 
situations. The dogs were trained to run to a given signal through a very 
a inforced by food (locomotor instrumental 


simple maze, each run being r d 
reflex). Further, tests were made of their ability to secure food by climbing 


onto a table, or going over a barrier. : M 

After all test responses had been established, and the conditioned reflexes 
had become stable and regular, more or less extensive ablations of the sensory 
cortex were performed. In two dogs, only the sensory arca of the left hemi- 
sphere was removed; in six dogs, bilateral sensory lesions werc performed; 
in one dog, first the left and after some time the right sensory cortex was 
ablated (pp. 309-310). . 

Results—After bilateral ablations of total sensory areas I and II, the general 
behaviour of the animals was unchanged. No tendency to hyperactivity or 
to stercoty ped movements was observed; the dogs found their way easily to 
the experimental room, they reacted normally to calling, and so on. 

In contradistinction to their gencral behaviour, proprioception was very 
much impaired. Very often they put the dorsal aspect of the foot to the 
ground, they slid on the smooth floor, they crossed their legs, ctc. All these 
defects were gradually compensated, but only partially. The placing reaction 
[not to be confused with the response of "placing? the forepaw on the 
food-tray—see below | w rently: abolished. 1 heir ability to go over 
a barrier and to climb ble was greater impaired, but improved 


gradually. 

When brought t 
adequately; they stooc 
nals. They manifested a Vt 
to the conditioned stimuli coi 
salivation, but they were comp 


putting the leg on the food-tray. : i 
This lack of instrumental conditioned reflexes lasted in various dogs from 


16 to 78 days. Then conditioned reflexes spontaneously reappeared, either 
1 h a series of abortive movements, or suddenly. 2-3 months 


gradually, throug t p : à 
after operation the movement of the leg was hardly different from that be- 


fore operation. E 
On the other hand, the locomotor conditioned reflexes were totally 
preserved and the animals easily found their way to the goal (p. 311). 


Discussion— V he problem arises, what is the cause of the abolition of the 


sas permar 
onto à tà 


o the experimental chamber, the dogs behaved quite 
] correctly on the stand awaiting the conditioned sig- 
a very clear and prompt general alimentary reaction 
'onsisting in turning towards the food-tray and 
letely unable to perform the movement. of 
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instrumental conditioned reflexes after sensory ablations, and why they 
reappear spontaneously after some time without any additional training. 
There is a vast body of evidence showing that, in all instrumental (“vol- 
untary") behaviour, the feed-back from the performance of movement and 
the posture of the extremity: plays a very important or, maybe. 
pensable role. By removal of the sensory cortex a gres 
feed-back connected with tactile and postur 
may be the reason why the animal is unable to perform the required move- 
ment. On the other hand, the general reaction towards food and the loco- 
motor conditioned reflexes are preserved because they are chiefly based on 
visual and labryinthine reception. As proprioception of the extremities is 
also represented in the so-called motor cortex, the animal probably relearns 
in his normal life, outside the experimental chamber, to perform “vol- 
untary" movements by means of his partially reduced feed-back, and when 


this has been achieved he is also able to perform the instrumental movement 
trained in the experimental situation. 


Which parts of the central mech 


, quite indis- 
at part of the sensory 
al stimuli is destroyed, and this 


"n. anism connected with proprioception are 
indispensable for performing the so-called “voluntary” movements is not 


quite clear. In particular, it remains to be elucidated whether sufficiently 
extensive cortical ablations, destroving all cortical representation of pro- 


prioception, would lead to the permanent and irreversible loss of instru- 
mental conditioned reactions (pp. 311-312). 


Observe how neatly these 


findings, and the authors’ interpretation 
thereof, parallel our own assumptions concerning the role of response- 
correlated stimulation in the establishment of ‘so-called habits. Note 
also the tendency of these authors to equate habit and voluntary action, 


thus tacitly repudiating the older idea that a habit is a relatively fixed 
connection or bond between a given stimulus and response which op- 
crates automatically, involuntarily, Without involvement of the rest of 
the nervous system, 


t : And it is likewise 
ings of Stepien & St 


instructive to compare the find- 
epien with I 


ashley’s (1934) conclusion that habits 
could Dt be localized at all, neurologically; that they were dependent 
upon mass action,” of the whole central nervous system. Here, in the 
work of Stepien & Stepien, we find that something a good deal more 
complicated than a stimulus-response bond, or A transit, is in- 


« "e 1 P n ite ee È A ga: s 
volved in a habit; but we also sce how specifically important arc the 
centers for the sensations 5 


1 aroused by performance of the response in 
question. 

Stepien & Stepien pertine 
executed mainly on the b 
prioceptive sensation prod 
their observations concer 


ently remark that when a habit can be 
asis of visual feedb 


; ack, impairment of pro- 
uces little 


I or no effect. Also interesting arc 
1 à ning labyrinthine stimulation. Ordinarily be- 
havior theorists do not think of the sensations generated by excitation 
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‘ f the semicircular canals and otoliths of the inner car as participating 
Jeo clean Ésesllsanbetn : es Ren 
a the feedback from and control over behavior. But it is well established 


that in all movement and postural changes involving the organism's 
bead, this type of stimulation can be highly important: it isa part of the 
lation resulting from movements of the 
head and body as a whole, no less than are visual, kinesthetic, tactile, and 
other forms of stimulation. This fact is well attested by the effect upon 
locomotion which removal of the 
jeldard, 1953)- Although feed- 
located directly in the muscles 
portant in the control of move- 


total response-correlated stimu 


postural adjustments and general 
labyrinthine organs is known to have (C 
sense which are 
ally im 


back from the organs of 
(1.c., the proprioceptors) is especi 
Ment, it is clear that various other sensory modalities play a con- 
and can, if necessary. often "take over" 


ne with what is, in effect, a 
and information from 
is equally prompt and 


comitant role in this connection 
We are here dealir 
1960, Chapter 8) 
provided it 


quite. satisfactorily. 
Servo-mechanism ( Mowrer, 
one source is just as good as another, 
precise. 

Somewhat anachronistically Stepien & Stepien 
the Pavlovian background of their work) to speak of the habits which 
they: studied as “instrumental alimentary conditioned reflexes” (type-I, 
from type-l, or classical, conditioned reflexes); 
“with which they are dealing is obviously 
S—R connection. 


than a one-way ~ 
he purest of so-called motor, or 


dependent upon the intactness of sensory 
his is destroved, the “reflex,” no less than 
ate. Thus, even the reflex (except 


continue (because of 


= distinguish them 
"ut the reaction mechanism witl 
al less simple 


something a good de 
that even t 


Actually, it is now known 
aia reflexes are also 
cedback therefrom; for if t 


a "habit," becomes highly incoordin : à 
When mediated by the autonomic nervous system) is no simple, ngle 
to conceive of habits in this arti- 


bond. So how much less plausible Is It 


ficial, unrealistic way! 


The Views of 

d Others 

ng and behavior adaptation which has 
a symphony. with many variations 
but with a basic form and stvle. 
ance of a contrapuntal theme cur- 
hologv. like a discordant ob- 
f whom seems to quite 


XII. A Dissonant Note: 

Woodworth, Hunt, an 
eory of learnir 
is book is like 
developments. 
aow take cogntz 


American psy 
advocates, no one o 


In general, the th 
been described in th 
and complementary 
How ever, we must t 
rently running through 
ligato, It has many individual 
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agree with any other; but two noteworthy attempts have recently been 
made to capture the essence of this movement and to give it succinct 
statement, which we shall here examine. 

In 1958 R. S. Woodworth published a book entitled Dynamics of Be- 
havior which, curiously, rejects “dynamics” in the customary sense of 
this term and places central emphasis instead on cognition—or what one 
might call “cognitics.” 


The book, says the author, is an claboration— 
both as regards theory 


y and scope—of those parts of his Experimental 
psychology (Woodworth, 1938) which dealt with “motivation, per- 
ception, and learning.” And it was also foreshadowed by the 1947 article 
cited earlier in this chapter. The thesis of Dynamics of behavior is that 
what living organisms learn, primarily, is not the solutions to problems 
(in the homeostatic sense); instead, says Woodworth they learn their 
environments! When one recalls that Woodworth was one of those 
psychologists who were responsible for reintroducing the organism 
into the S—R formula, thus making it read S—O—R, it is surprising 
how little attention he is now disposed to give to the organism, as such. 
Part of the trouble a ises, he savs, from the fact that “Fundamental 
laws of learning and motivation have been obtained from the animal 
laboratory" (p. 12); and these need to be re-examined in the light of 


what we know from the study of human sensation and perception. 
It is not easy to get hold 


Woodworth would present 
the following quotation is per 


. quickly and firmly, of the ideas which 
as a substitute for prevailing theory; but 
haps as close to a résumé as one can find: 


Without going back 


! far into the historv of the subject we shall con- 
sider two oppose 


à d theories—one of which is accepted by many psychol- 
ogists at the present time; the other is emerging and deserving of a radical 
presentation and support. The first may be called the need-primacy theory 
and the other the bebavior-primac y The need-primacy theory holds that 
all behavior is Motivated directly or indirectly by the internal needs of the 
organism, such as hunger and thirst. Sex is ‘regarded as a need to escape 
from certain internal tensions, and the need to c cape from pain and ex- 
ternal injury must also be included. The behavior-primacy theory holds 
that all behavior is directed primarily toward dealing with the environment. 
Even eating and drinking, which certainly rid the organism of needs, can be 
regarded as ways of dealing with the env ironment and. securing positive benc- 
fits and satisfactions; i.e., as forms of behavior w hich secure positive good 
rather than merely escape from harm, No matter which of these theories 
has the greater emotional appeal to the reader, the scientific question is 
which has more predictive value. Which one better covers the ground and 
is more systematic and parsimonious? At first sight the need-primacy: theory 
would be the more parsimonious and systematic if it could possibly be 
stretched to cover the play motives of children and the absorbing interests 
of adults. We should have to show how these motives and interests could 
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possibly be derived from the primary needs; thus we could “explain” the 
motives. But could we predict them for the individual or for the social 
group without assuming additional primaries besides the organic needs? 
Two svstems which clearly belong in the need-primacy class are the 
psychoanalytic "stem of Sigmund Freud and the behavioristic system of 


Clark Hull (pp. 1017102). 


In one sense, the whole of the present book is an answer to Wood- 
worth's challenge to the predictive capacity of homeostatic theory; 
and many of the problems which he raises have been dealt with spe- 
cifically in Chapter 6 (see also Mowrer, 1960, Chapter 10). Moreover, 
the present writer has elsewhere (Mowrer, 1959) reviewed the Wood- 
1 out some of its palpable weaknesses. These 


worth book and pointed 
Instead, let us attempt an over-all ap- 


need not be re-examined here. 
praisal, in the form of two major points: 

1). Despite many sallies against it, Woodworth knows that homeo- 
stasis cannot be totally disregarded in the analysis and explanation of be- 
havior, This point is implied or expressed in various places: eg; “We 
at the organic needs are derived from the tendency 


are not pretending th i 
organic needs are autonomous” 


to deal with the environment. The 
(p. 128). But perhaps the most interesting elaborations of the relation- 


ship between homeostasis and learning occur in the following passage: 


as the advantage over Freud's of taking much more ac- 
count of the effects of learning. What he calls primary reinforcement is 

isfaction of a primary need such as hunger for food. More exactly, 
lull as the cessation or reduction of the need... . When 
an act leads promptly to such primary reinforcement, learning occurs. The 
act that is reinforced becomes conditioned to the need and to the external 
situation so that when the same need and situation occur again, the same act 
is likely to be made in response. Consequently, by repetition with rein- 
forcement, a habit or mechanism is built up W hich deals with the situation 
in such a way as to satisfy or reduce the need (p. 110). 


Hull's system h 


the sa 
it is defined by H 


„is an excellent statement of Hull's scheme for account- 
ing for both T horndikian learning (the act that is reinforced becomes 
conditioned to the need") and Pavlovian conditioning ("and to the ex- 
ternal situation"), which we have considered at length in Chapter 3 and 
also earlier in this chapter. But then Woodworth makes this additional 
comment, as if by way of further exposition of Hulls views, which is 
more accurately an extension of Woodworth's own earlier (1918) think- 


Here, surcl 


ing: 


mechanism itself acquires some secondary incentive 


More than that—the me ase ‘ s y 
hanism consists in going to a certain place where food 


value. Suppose the mec 
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is found; this place and the act of going there will serve as reward für i 
number of trials even if no more food is found there. Or, if the drive Is 
changed from hunger to thirst, the animal will go there for water which he 
has never found there. This place has become a “good” place. similarly, A 
place where an animal has been shocked has become a bad place, a place 
to be avoided; the place to which he goes and where he gets no shocks has 
become a good place. (Our way of stating the facts is somew hat different 
from Hull's). There are many similar results from the animal laboratories. 
A stimulus that just precedes the obtaining of food—like the click of the 
pellet-delivering machine in the Skinner box—becomes something worth 
working for even after the primary reward has ceased (pp. 1107111). 
Woodworth obviously concedes these (homeostatic) “facts.” And 
he casts them in such a way as to bring his statement of them very close 
to our position (cf. also his discussion of “sensory feedback,” p. 139, in 
his chapter on “The Control of Movement”). But he then, so to speak, 
throws it all aw ay by citing some very 
search on secondary 
devaluate the import 


incomplete and antequated re- 
reinforcement which he believes enables him to 

ance of this type of analysis. 

2). From the strategy 

his main point: that le 

environment, 


just described, Woodworth goes on to make 
arning is essentially: cognitive, a learning “of the 
rather than of specific responses of any kind (be they 
behavioral or emotional). Chapters 9, 10, and 11 of Dynamics of Be- 
havior are all entitled “Learning the Environment” (with appropriate 
subtitles), and the first of these epitomizes his argument thus: 


In accordance with 


: our general definition of behavior as a “dealing with 
the environment," 


several chapters on "learning to deal with the environ- 
ment" would be expected, for certainly a great deal of learning is necessary. 
If, instead, this series of chapters is headed learning the environment, some 
explanation and justification are in order. Such a phrase as “learning a person 
or thing or place" has an unfamiliar, awkward sound, though the awkward- 
ness is gone if we say “getting acquainted with” instead of “learning.” No 
one can deny that getting acquainted with the environment and with the 
objects in it is a form of learning and should be included in any general in- 
vestigation of the learning process. Our choice of chapter headings is in- 
fluenced by the conviction, several times expressed already, that the motor 
behavior of human beings (at least) is guided by the perception of objective 
facts. Accordingly, learning to deal with the environment depends on €x- 


ploration and perception of the environment, with c 
has been perceived to influence l 


nough retention of what 
ater behavior (p. 221). 


Here, in a somewhat round- 


about w ax. 
real problem. We ; 


have previously indic 
that whereas our system is very s 
of behavior once it is selected 
of how one act rather than 


Woodworth is pointing eee 
ated (Chapter 7, Section IX) 
atisfactory: for explaining the guidance 
and initiated, it leaves open the question 
another gets started. And it has also been 
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»ointc T a e 
pointed out that the phenomenon of conditioned sensations, 1n the form 
ably intimately associated with 


of imagery (or memories), is probs 

be discussed in detail until later 
(Mowrer, 1960, Chapters 5-7). Now it is obvious what Woodworth has 
learning to know a person, surely, is 
c sort of that person when 


his name or some other identi tic is mentioned. This, as 


we have already noted, is a real phenomenon, but we see it as supple- 
menting rather than supplanting the process of affective conditioning, 
as it relates to reward and punishment (or homeostasis). In stressing 


the primacy of cognition, Woodworth seems to be reverting to pre- 
its emphasis upon consciousness and 


an objective, behavioral base 
affectivity and then 
systematic con- 


he answer, This conjecture will not 


in mind in the above quotation: 
or memory of som 


learning to have an image 
fying characteris 


behavioristic psychology, with 
Conscious states. per se. ‘If one starts with 
and then, as occasion demands, introduces first 
Imagery, one has, it seems, a far 
instead starts 
all consider 
regards very hi 
d although Tolman has use- 
"come out," he has 


sounder and more 
“at the top.” 
the theoretical position of 
ghly—and so shall 


Ceptual scheme than if one 
In the next chapter we shi 
E. C. Tolman, which Woodworth 
we! But it is now generally recognized that 
fully indicated w here a behavior system ought to 
Dot been very helpful in showing exactly how to get there. The same 
criticism: is also applicable. it seems, tO Woodworth's approach. Pro- 
fessor Woodworth undoubtedly docs us à substantial service in noting 
the neglect of cognitive processes in prevailing behavior theory; and 
5-0 critique of cha theory his work is highly valuable. If, however, 
Da substitute for existing behavior theory, 
less certain. 
been scrutinizing the field 
erence to human develop- 


mmarized his impressions 


his vi 

oe are presented as 
their value and validity are considerably 
ep MeV. Hunt has 


“ith special ref 
was recently su 


For some time, now. J 
Of motivation and learnin 
mental psychology: and he | 
(Hunt, 1959) as follows: 


gw 


Jominant theory. it is claimed, first of all, that “all be- 
at the aim or function of every instinct, defense, 

liminate stimulation or excitation within 
re driven, first. by those so-called pri- 
homeostatic imbalances or needs. . . - 
forms of intense and painful ex- 


According to our c 
havior is motivated," and th 
action, or habit is to reduce OF e 
the nervous svstem. . <- OT nisms à 
Mary, inner stimuli w hich arise from © 
Organisms are driven. second, by various 
ternal stimulation (273)* T. 

lt is also assumed, as the proposition that 
Plies, that the organism would be mac e un 
Outer stimuli (p. p- 

The dominant theory h 


“all behavior is motivated" im- 
less driven by either inner or 


as been a conceptual edifice of large demensions 
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and considerable detail. It has provided a plausible account of both person- 
ality development and social motives. The experimental facts of homco- 
stasis and of conditioned drive and fear are sound. Nevertheless, it has be- 
come more and more evident in the past 10 years that some of the explanatory 
extrapolations contradict facts and call for reinterpretation (p. 7) 

The first of the assumptions to be called into question is the one that all 
behavior is motivated and that organisms become inactive unless stimulated 
by homeostatic need or painful stimulation or conditioned stimuli for these 


(pi). 


Elsewhere in this volume we, too, have advanced reasons for believing 
that there is no one-to-one relationship between drive and activity (sec 
especially Chapter 6 and Section V of the present chapter). But our 
reservation has taken the form of merely noting that drive can bc 


present without activity. Hunt maintains that the reverse also is truc. 
He says: 


Let us note that such observations do contradict our 
ganisms will become inactive unless driven by homcostat 
ful stimuli, and give up this ancient € ) 
substance to which motive must be 
embrace the thermodynamic conc 
energy exchange which exhibit a 
lating effect, but not an initiating effect (p. 9). 


The point of mentioning these evidences of spontancous activities of 
organ systems is merely to help inter for good the notion that activity of 
living systems requires homeostatic need or painful external stimulation and 
to foster the idea that to live means to be active in some degree. 

This idea of activity being intrinsic in living organisms has implications 
for our conception of reinforcement. It makes it unnec v to see all 


d * a matter of cither reducing or avoiding stimulation which is im- 
p ed T the assumption that organisms become inactive unless stimulated 
pP- 10). 


ssumption that or- 
ic needs and pain- 
ireck notion that living matter is inert 
imparted by extrinsic forces. We can then 
eption of living things as open systems of 
ctivity upon which stimuli have a modu- 


The difficulty is that when one leaves the homeostatic model and 
averts to org: 


anisms which behave “spontancously,” one also leaves 
the realm of orderly principle, prediction, and control. Perhaps this is 
as it sbould be. Perhaps the science of behavior is, at best, severely 
circumscribed and that the fact of spontaneity is the reason for such a 
state of affairs. Yet this is not w hat Hunt, Woodworth, and related 


riters seem to be say ing. They seem to be implying, instead, that some 
sort of orderliness or lawfulness is possible à thi 


neitv." This remains to be 
homeostatic model already 
plishment, as Hunt (as well 


within this realm of “sponta- 
demonstrated. On the other hand, the 
has a long record of substantial accom- 
as Woodworth) readily grants: 
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There is still a place for drive reduction. It is clear that under conditions 
of homeostatic need and painful stimulation, and perhaps under circum- 
stance when the conditions of stimulation are changing with too great 
both animals and persons learn techniques and strategies leading 
cation or reduction in external stimulation. The evidence that led 
is as convincing as ever. More- 
atic need, animals and men 


rapidity 
to grati 
Thorndike to formulate the “law of effect? 
over, in association with reductions in homecos 
may also learn cathexes or emotional attachments. The facts referred to are 
those highly familiar in secondary reinforcement (Pavlov, 1927; Hull, 1943) 


(p. 11). 


"The objections and reservations. with respect to the homeostatic 
principle which Hunt then proceeds to cite constitute important and 
useful challenges to this principle; but, as noted in the present writer’s 
review of Woodworth's book, "thev have neither unhorsed it nor led 
to à rival conceptual scheme of comparable cogeney" (p. 130). Granted 
that homeostatie theory is presently laggard in its treatment of cognitive 
functions, this is not to sav that the thing to do is to scrap the theory, 
because it is in this one respect undeveloped. Cognition and 
anchored in the good bedrock of homeostasis 
anything more than scattered, unsys 
tematic observations and data. They do not give us a comprehensive, 
powerful, unified theory. Our faith is that the homeostatic model can 
and indeed sust be extended to include cognitive phenomena, not only 
because of their independent reality, but also because the homeostatic 
model has itself reached a stage of development where cognitive proc- 
esses are logically demanded. This is a major task and objective of 
Learning Theory and the Symbolic Processes, the sequel to the present 
work. However, in the remaining four chapters of this book, we shall 
make no further attempt to deal with this aspect of the problem (see, 
however, Chapter 9. Section V =) but will instead proceed on the as- 
sumption that the best way to make progress is to continue to build 
on the substantial foundation which homeostasis already provides, 


rather than launch off or 
abortive form of analysts. 

r 6 for reference to recent studies of the reticular formation 
1 its bearing on the question of behavior spontaneity (also 


simply 
“spontaneity,” unless 
seem incapable of generating 


1 some seemingly casier but, in the long run, 


* See also Chapte 
(“arousal center") anc 


Mowrer, 1960, Chapter 7)- 
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Reflexology and Psychologi 
extremes; and controversy, 
between the proponents th 


al “field theory" have represented pa 
not infrequently heated, has often occurred 
creof. Revised two-factor theory takes an 
intermediate position designed to effect, not merely a compromise 
and uneasy peace, but a true synthesis and logical integration. 

We shall here consider, first of all, the more radical version of field 
theory developed by Kurt Lewin and his students and will show how 
naturally it articulates with the systematic views which are presently 
under discussion. We shall then survey some of the forerunners of 
field theory, with special reference to the work of E. C. Tolman and 
his students on the subject of place versus response learning, as it has 


been called. We shall consider current researches on the problem of 
behavior Variability in a more or less “ n 
ine some of the svsten 
see that in his las 


roe open-field" situation and exi l 
atic implications thereof, And finally, we shal 
ast book Clark Hull, throwing pride and past objec- 
tions to the winds, himself attempted a kind of behavioral analysis 
which is manifestly “field theoretical,” 

Were it not that the principles which 
possible derive mainly from the 
temporary situation might seem 
field theorists. Perh P 
truth has existed or 


a make this emerging synthesis 
stimulus-response tradition, the con- 
; to constitute quite a victory for m 
APs, as 15 So often the case. an important part of the 
n both sides of this argument. 


l. Recapitulation 


However, before proceeding 


1 4 i ʻi re. o 
with this chapter, it will be w ell t 
pause for a moment 


> in our upward “climb,” and look back and about 
306 
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us to see what may have come clearly into view which we have 
. not at all. 

distinguish. between metabolic 
able reactions which we call 
by acknowledging the latter 


an adequate psychology of 
P D. 


previously seen only dimly or perhaps 

We now see how essential it is to 
(or primary) drives and those condition 
emotions. We see, indeed, that it is only 
that we can have anything remotely like 
learning and performance. 

Thorndike had little or nothing to say 
edly he felt he did not need them in his system. He dealt instead in 
terms of hypothetical bonds supposedly established (by reward) or 
destroyed (by punishment) directly between the metabolic drives 
(hunger, thirst, etc.) and succe ful or unsuccessful consummatory 
(instrumental) behavior. Elegant in its simplicity, this scheme never- 
theless had some fatal defects. 

Pavlov likewise had no use ( 
fered from Thorndike in holding th 
not because of changes in the “strength” c 
UnCS—R) bonds, but because of changes in the connection or bond 
in question and some zew stimulus (CS). Again 


also had certain crucial short- 


about the emotions; undoubt- 


literally) for the emotions; but he dif- 
at biological adaptation occurs, 
of drive-response (or 


between the response 
clegantly simple. pavlov's system 
comings. 


. i — Mm u ik n Day , 
Hull, as we know, tried to "integrate Thorndike and Pavlov, but 


Thorndikian. While he sometimes acknowl- 
drives or emotions (viz. fear), he as- 
a matter of bonds being formed 


his scheme was basically 
edged the reality of secondary 
sumed, here again, that learning was b 
between these drives and drive-reducing overt actions. 
Our assumption, bv contrast, is that. the emotions play a central, 
] anges in behavior or perform- 


indeed an indispensable, role in those ch FOU A , 
ance whieh are said. to “learning. The emotions are in- 


volved, first of all, in that they 
Fear, hope, relief. and disappoin 


most readily 
endent and/or response-dependent stimuli, 


represent 
are, strictly speaking, what is learned. 
tment—these we assume are the re- 
detions whic are and importantly conditionable; and 
ioned, to indep 
and contr 


once condit 
they then guide 


adaptive manner. 3 
Thus far, major attention has been given to the changes in behavior 


when emotions get conditioned to response-correlated 
stimuli. The reason for this is that our principal objective has been 
as a counterpart to the "new theory of punishment—see 
w, and improved, way of thinking about the 

But, in the present chapter, we shall discover 


ol performance in a generally. sensible, 


that. occur 


to develop ( 
Chapters 2 to 4) 3 ne 
phenomenon of “habit. 
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3 T 4 Y Rf EN TT NU TII i o 
that the foundation thus laid is also highly satisfactory for accounting 
for so-called place learning or “field” behavior. 

Here it remains only to be said—or, 


rather, re emphasized—that, 
whatever its virtues and strengths (which were considerable), Be- 
haviorism's rejection of emotions was a 
present conceptual scheme is valid, Bore 
central importance to a theory of learning and behavior modification 
and control. This position contrasts radically with the teaching of 
one of the writer's early te 
"wasted reactions" ( Mever, 1922, 1933 ; 

There has been a widespread tendency in Western civilization to 
look upon “the emotions” with a certain distrust and contempt and 
to elevate “the intellect" (reason, logic) high above them. If the PISS 
ent analysis is sound, the emotions are of quite extraordinary im- 
portance in the total economy of living organisms and do not at all 
deserve being put into Opposition with "intelligence." “Phe emotions 
are, it seems, themselves a high order of intelligence. ‘The full mean- 
ing and justification of this Statement will unfold gradually in this and 
later chapters (see also Mowrer, 1960). 


major stumbling block. If the 
the emotions are of absolutely 


achers to the effect that emotions are 


Il. Field Theory as Developed by Kurt Lewin 
As Deutsch (1954) and 
developed by I 
theorizing of 


Others have pointed out, ficld theory, as 
swin and associates, is largely an outgrowth of thc 
19th. century physicists. Instead of thinking of a discrete 
cause producing a correspondingly discrete. effect, these men—in- 
cluding Maxwell, Faraday, and Hertz —found that they could formu- 
late more inclusive laws and write more powerful cquations if they 


made reference to fields, notably gravitional and electromagnetic 
fields. 


One of the first im 
Was to produce 
flexology. Inste 
bonds, either 


pacts of field-theoretical thinking in psychology 
à conception of behavior antithetical to that of rc- 
ad of secing behavior as made up of isolated S—R 
native or acquired, the field theorists (including Gestalt 
psychologists) took the position that the individual alw ays reacts as 
a totality, as a more or less well-organized entity in response to a 
total situation in so far as it is perceived and interpreted by the indi- 
vidual. Moreover, as Deutsch observes: 

The | field-theoretic 


al | explanation of behavior assumes that all behaviors 
have directional char 


acteristics. Hence, it is concerned with the purposes 
which underlie behavior and the goals towards or away from which be- 
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havior is directed. There is little direct interest in tools or mediating processes 
per se apart from the interest in how they are learned or acquired and in 
how desirable and effective they are considered to be by the individual 
(p. 183). i 

In other words, an act or action is not something which is auto- 
matically "run off," once the adequate stimulus has set a given cause- 
cffect sequence in motion; an action is something which is being 
continually guided and corrected by additional stimulation or "in- 
formation” which is impinging upon the organism. Passing reference 
has already been made (Chapter 7) to the concept of “feedback,” and 
it is considered more fully in another connection (Mowrer, 1960, 
Chapter 7). Suffice it to say here that it is closely related to field 
theory. In both cases it is sumed that a given behavior sequence, 
once set in motion, is not incluctably determined but is, rather, sub- 


ject to continuous modification on the basis of the “meaning of the 
as it is sensed or perceived moment-to-moment. In other 


situation,” 
words, behavior is a continuous, on-going function of the informa- 
tional feedback from all senses, internal and external; it is, in a word, 
a function of the total psychological field. 

Furthermore, as already implied, the effects of a given constellation 
of stimuli, a "situation," or a "field," upon an individual cannot be 
predicted or even understood without knowledge of the individual and 
what he brings to the situation. To quote Deutsch again: 


Lewin's point is that to develop psychological laws we must deal with 
psychological processes—the perception of the external event, not the event 
itself; behavior, and not its effects by themselves. Thus, we ought to be con- 
cerned with the psychological impact of a “reward,” cg. whether it is 
perceived as a "bribe" or as a "sign of accomplishment," rather than simply 
with the external situation of reward. In essence, one has to deal with what 
exists psycbologically, what is real for the person being studied (p. 184). 


That is to sav, even though one has taken the total situation into 
account, at least as far as its stimulus aspects are concerned, one must, 
in addition, consider what Tolman (see Section IV) has aptly called 
intervening variables. External events or stimuli influence and deter- 
mine behavior, not in a direct, physicalistic sense, but through the in- 
tervening action of the meanings they have acquired for the individual 
as a result of prior xperience, i.c., through learning, either immediate 
or vicarious (see Krecheysky, 1932; also Mowrer, 1960, Chapter 6).! 

Enough has now been said about psychological field theory, partic- 


!For further discussion of what he terms the "cognition versus association 
issue" in psychology, see Spence Crostb, pp. 241 ff). 
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ularly as developed by Kurt Lewin 
a comparison with revised two-f. 
gruences are striking. Both 


and his students, to make possible 
actor learning theory. The con- 
arc agreed that a rigid S—R, reflexological 
interpretation of the behavior of higher organisms is not feasible. And 
both are also agreed that, at the very least, one must have a two-step 
conception of situation-behavio c 
mental and response-produced, c 
which (along with organically 
determiners of overt r 


r Sequences; stimuli, purely environ- 
licit meanings, motives, interpretations 
given drives) are then the immediate 
action, not the "raw" stimuli themselves. I 

But two-factor theory differs from field theory in this respect: ip 
does not find the concept of reflex and conditioned reflex entirely 
useless. It agrees that these concepts, applied to overt, behavioral re- 
sponses, are misleading and inadequate. It holds, however, that Hn 
meanings, emotions, secondary: Motives, and interpretations which, aR 
a result of learning, become attached to particular stimuli or situa- 
tions are basically reflexive (involuntary, autonomically mediated) 


: : AU ep at 
and that the best paradigm for the learning here involved is th 
of conditioning, or 


sign learning, Associationistic psychology, m- 
cluding both Conditioning and habit theory, has in the past often 
seemed to be diametrically opposed to field theory, Two-factor theory 
offers a possible avenue of reconciliation.” It agrees with field theory 
and disagrees with associationism in holding that the concept of re- 
flex, be it in the guise of “habit” 7 


or conditioned reflex, does not give 
anything like an adequate general explanation of behavior. But it dis- 
agrees with, or ar least goes bevond, field theory and espouses re- 
flexological thinking in holding that it is precisely through condi- 
tioning, or associative shifting, that stimuli take on their inner mean- 
ings for the individual, that signs and situations acquire their signifi- 
Cances. Meanings, it is held, Constitute for the individual a sort of 
inner, subjective field; and it is this on-going, ever-changing motiva- 
tional state Which, moment by moment, modi 
behavior. X 


s, controls, and directs 


*As if anticipating the possibility of such a reconcil 
served: "During recent ye. \ 


have come to char 


a conflict between two camps, the one commonly called stimulus-response or 
stimulus-response-reinforcement theory and the other gestalt, cognitive or, most 
generally, field theory. The lines of demarcation are not entirely distinct” (p. 
317). “| Burt] there is no reason to expect that direct comparison of learning theory 
and field theory will be easy .. 7 (p. 318). And Fstes concludes that “it did a 
p this disparity at a level of broad generalities (p. 341). It 


tion, Fstes (1954) has ob- 
i i i "ori mselves, 
ars reviewers and text writers, if not theorists themselv 


: Arninge in terms of 
acterize, or dramatize, the psychology of learning in term 


seem possible to clear 
u 


; : s ip dp Pun ra- 
is believed that the presenr theoretical framework provides the basis for an integ 


ton at just such “a level of broad generalities.” 
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At least at one stage, Lewin's work took a different turn. He at- 
tempted to organize his theorizing around a sort of geometric, or 
spatial, model called topology. “Vectors” and “valences” were sup- 
posed to articulate the individual with his “psychological space" or 
environment; and these, according to their intensity and sign (posi- 
tive or negative), constituted. the "field" in which the individual re- 
acted and behaved.? Two-factor theory also sees signs and situations 
as having meanings which can properly be interpreted as positive or 
negative: if a sign is “negative” it produces an increment in emotional 
tension, and if it is "positive" it produces a decrement in emotional 
tension. But beyond this the agreement with topology is tenuous. 
Lewin, despite his acknowledgment of meanings, seemed to give con- 
siderable potency to external, environmental, or field forces as the 
actual determiners of behavior. In contrast, two-factor theory cm- 
phasizes an internal, rather than an external, “field” as the dynamic, 
energy-producing, causally effective source of behavior. Ordinarily, 
the amount of energy involved in the stimuli impinging upon living 
organisms from without is miniscule and can be detected only by 
the most refined and sensitive types of physical apparatus. But, some- 
What in the manner of a radio receiving set, living organisms take 
these minute and, as such, causally ineffective stimuli and convert, 
or amplify, them into forms of psychological energy which are quite 
powerful. A stimulus which is very weak in its own right—for 
example, a whispered or written word—may bring either overwhelm- 
ing fright or joyous relief to an individual; but it is surely the “am- 
plified,” interpreted, conditioned inner reaction to the stimulus and 
not the stimulus itself that produces these important motivational 


consequences. t 
Perhaps, for the reasons just cited, we can see why the topological 
formulation of field theory was not very successful. Says Deutsch: 


Although Lewin's "topological" and "hodological" concepts are not much 
more than diagrammatic representations of his brilliant theoretical and ex- 
perimental insights, it would be foolish to neglect the suggestive value of his 
imaginative attempt to develop a geometry suitable for psychologists. He 
pointed out the need for a new mathematics based on axioms different from 
those of the mathematics developed for the physical sciences. He indicated 


“The term valence, borrowed from chemistry, here implies a direct force of 
attraction or repulsion, vector, on the other hand, comes from mathematics and 
was used by Lewin to imply the net effect, 
two or more valences simultancously impinging upon the individual. 

‘Lewin himself (1942), in an article on "Field Theory of Learning,” acknowl- 
edged that valences “change w ith experience” thus implying that it is not the 
stimulus itself that supplies the "valence" but a learned reaction thereto. 


algebraic summation, or resultant of 
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i 3 $i etry adequate to 
some of the properties that would be required of a geometry ind eae 
handle psychological space and he stimulated a more widespread in : 
in the development of such a geometry (p. 199). 


More relevant to and compatible with two-factor theory nre ped 
of Lewin's earlier hypotheses concerning tension systems. These ar 
internal to the organism and were thought of as producing (1) loco- 
motion or movement away from stimuli or situations that auguent 
tension and (2) locomotion or movement tozeard stimuli or situations 
that diminish tension. This mode of thought, which is w holly PUn 
patible with two-factor learning theory, prompted. Lewin meee 
students to do some of their most ingenious research: the study p 
completed and uncompleted tasks (Zeigarnick effect), detour mig ea 
satiation, frustration, substitution, levels of reality, and so on. Much 
of this research, it is believed, can now be more felicitously dies 
preted in the terms of two-factor theory than in those rec 
cally employed by Lewin; and in making such a translation n "m 
lates these investigations to other psychological systems, from whic 
they originally seemed quite remote. ' AT. 

Because this book introduces the concept of “hope, presumably o 
the first time, into stimulus-response theorizing, it is interesting. E 
discover how fre ely it has already been employed in field-theoretica 
considerations. Says Deutsch: 


An emphasis on th 
in contrast to an e 
processes has sever: 
the 


€ psychological explanation. of psychological S 
‘planation in terms of physical or peur phase uA 
al important consequences. The scientist who works hee 
assumption that psychological phenomena. are rcalities, even if ur 
cannot be expressed in physical terms or located in physical space, is NE 
apt to investigate them. Recognizing that people in every day fairs use Mu. 
concepts as "hope," "desire," "action," "ability," "ought," eten, to g i 2. 
acterize others as well as themselves. (and acknowledging that these p p 
nomena have reality in their own right), he is more likely to feel that a 
concepts of this everyday: psy chology are legitimate and Important AEN 
for investigation by a scientific psychology (Heider, 1953). That is, 

will want to know under wh f 
will say he is “hopeful” 
that another person is * 
himself with the 


at conditions a person with given characteristi ? 
and under what conditions this person. will n 
hopeful." In other words, he i more apt to i aot 
psychological phenomena which occur in everyday ig te 
cluding the concepts of everyday psychology, than is the scientist w jm T 
that he is not being scientific unless he can cither say "hope? is rea Nac 
specified pattern of neural excitation in a certain region of the un or 
"hope" is really a certain pattern of bodily movements w ied M RIA hn 
pressed in physical terms, This is not to assert that the psy PRU Dem at 
employs a psy chological approach in theorizing will be Aide Rd 
the level of description of the phenomenal properties of psychological e d 
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He will want to know how “hope” can be characterized in terms of the 
constructs of a psychological theory. Lewin, in his writings, continuously 
stresses that the understanding of causal relationships requires one to pro- 
ceed from the more accessible, phenomenal properties of psychological 
events to their underlying dynamic or geno-typical properties (pp. 183-184). 


Chapter IV of Lewin's Dynamic Theory of Personality (1935) is 
entitled, “The Psychological Situations of Reward and Punishment”; 
and it might be expected that here Lewin himself would have estab- 
lished points of contact between his thinking and that of Thorndike, 
Pavlov, and others whose writings have provided much of the back- 
ground for two-factor theory. However, the chapter is largely de- 
voted to showing how reward and punishment are interpretable in 
topological terms; and there is no reference to Thorndike, Pavlov, 
or Hull or to the major ideas with which they were concerned. 

In two other papers, “H ield Theory of Learning" (1942) and “Field 
Theory and Learning" (1951). Lewin more nearly articulates his partic- 
ular concepts with those of stimulus-response theorists. Here he indi- 
cates two types or forms of learning: learning as it relates to changes 
in cognition, or "cognitive structure," and learning as it related to 
motivation (cf. Mowrer, 1960, Chapter 5). But even here a language 
is used which is foreign to most learning theorists, and not much of 
an integration is achieved. However, as indicated in the preceding 
pages, Lewin's formulations are by no means unrelatable to a modified 
stimulus-response psychology; and it appears that the two types of 
svstem can now be merged into a theorctical structure which is both 
more comprehensive and stable. 

In his recent review and critique of Lewinian theory, Bronfen- 
brenner (1951) has made some summarizing statements which can be 


usefully quoted. He say 


Lewin emphasizes that personality can be understood only if it is viewed, 
not in isolation, but in relation to the field in which it operates. This leads 
directly. into the most familiar aspect of Lewin's theory—his emphasi on 
the importance of the environmental field in terms of physiomathematical 
concepts—such as valence, vector, barrier, detour, and the like (p. 211). 

[But] even such a concept as valence, which upon first consideration 
appears to be an attribute of the object, is seen upon analysis to arise from 
needs or tensions within the perceiving organism—the person: 

“The close connection . . . between the perceptual field and the course 
of the process must not let us forget that the forces which control the 
course of the process remain without effect or simply do not arise when 
no psychical energies are present, w hen there exists no connection with 
tense psychical systems w hich keep the process in motion" (Lewin, 1935, 
PP- 50-51) (p- 212). 
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We are told that there are 
the individual and his envirom 
come, and whither they 
place for a more 


systems of forces acting within and between 
nent, but what these forces are, whence they 
impel, remain unanswered questions. . . ne 
comprehensive discussion of the internal structure of per- 
sonality” was evidently not in this world (pp. 214-215). 


It is believed that the concepts of hope and fear, as developed in the 
present volume, provide the “internal structure" that Bronfenbrenner 
is calling for and, at the same time, articulate the organism with the 
stimulus environment, or "field," which was of such special interest 


to Lewin and which he studied so productively. 


Ill. Adumbrations of Psychological Field Theory 
and Two-Factor Theory (Freud and Craig) 


As already indicated (in Chapter 8), Sigmund Freud could hardly 
be called a learning theorist, in any formal sense of that term, yet 
one finds in his Writings two concepts which are more than a little 
reminiscent of "hope" and “fear,” as emploved in this and preceding 
chapters. Although Freud hardly ever used the term “learning,” he 
frequently referred to cathexis as a form of “interest” which is at- 
tached to or invested ina given object or activity because that object 
or activity has been associated with pleasure; and he referred to anti- 
cathe š the state of mind engendered toward a given object or ac- 
tivity as a result of its having been associated with frustration, dis- 


appointment, or pain. A cathected object w 
tends to seck or 


an individu 


as one which an individual 
approach, and an object with anticathexis is one which 
al tends to retreat from or avoid. Similarly, a cathected 
activity is one which an organism tends to repeat; and an activity 
which has taken on anticathexis is one which an organism tends not 
to repeat. Freud was never very explicit about these concepts, beyond 
saving that cathexis involves a "charge of energy” and that anticathexis 
involves a “counter-charge” (1935. pp. 314-315). But without doing 
violence to his thought, one can interpret them as representing, Jus 
spectively, Conditioned tension decrement and conditioned tension in- 
crement, i.e, secondary rey ard and secondary. motivation, With such 
a translation, Freudian theory is readily relatable to the integrated 


version of field theory and habit theory which is represented by two- 
factor learning theory, 


ee 2 è ^ : T. conte rary learn- 
“For a more detailed discussion of Freud's influence upon contemporary learr 


ing theory, see Hilgard (1956, Chapter g). 
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Also of considerable historical importance and influence is a paper, 
entitled “Appetites and Aversions as Constituents of Instincts,” which 
was published in 1918 by Wallace Craig. Tolman (see Section V) has 
repeatedly acknowledged a debt to this paper, and in it we see one 
of the first stirrings of dissatisfaction with the reflexology introduced 
by Behaviorism and a clear intimation of the concepts of hope and 
fear. The gist of Craig’s thesis is given in the opening paragraph of his 


paper: 

The overt behavior of adult animals occurs largely in rather definite 
chains and cycles, and it has been held that these are merely chain reflexes. 
Many years of study of the behavior of animals—studies especially of the 
blond ring-dove (Turtur risorius) and other pigeons—have convinced me 
that instinctive behavior does not consist of mere chain reflexes; it involves 
other factors which it is the purpose of this article to describe. I do not 
deny that innate chain reflexes constitute a. considerable part of the in- 
stinctive equipment of doves. Indeed, I think it probable that some of the 
dove's instincts include an element which is even a tropism as described 
by Loeb. But with few if any exceptions among the instincts of doves, 
this reflex action constitutes only a part of each instinct in which it is 
present. Each instinct involves az element of appetite, or aversion, or both 


(p. 91; italics added). 


One observes, both from the title of Craig’s article and from the 
foregoing quotation, that his interest is centered upon “instincts,” 
rather than upon "habits"; but it should be remembered that at the 
time Craig was writing the former term was often used very broadly; 
and on a later page (p. 94) Craig indicates that, in his own thinking, 
"instincts". often include learned elements. (Others—Holt, 1931; 
Wheeler, 1928—have of course held that “instincts” are largely or 
wholly the products of learning). Therefore, it is of interest to ex- 
plore Craig's paper somewhat more thoroughly. He continues by 


saying: 


An appetite (or appetence, if this term may be used with a purely be- 
havioristic meaning), so far as externally observable, is a state of agitation 
which continues so long as a certain stimulus, which may be called the 
appeted | compare Freud’s term, cathected | stimulus, is absent. When the 
appeted stimulus is at length received it stimulates a consummiatory reaction, 
after which the appetitive behavior ceases and is succeeded by a state of 


relative rest. 

An aversion is a state of agitation which continues so long as a certain 
stimulus, referred to as the disturbing stimulus, is present; but which ceases, 
being replaced by a state of relative rest, when that stimulus has ceased to 


act on the sense-organs (p. 91). 
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And later Craig adds: 


Appetitive behavior in vertebrates is evidently a higher development of 
what Jennings (1906, p. 309) calls the positive reaction in lower organisms; 
aversive behavior in vertebrates corresponds to what Jennings (p. 301) calls 
negative reactions (p. 95). 


Craig is obviously caught in a dilemma: he wishes to remain within 
a behavioristic framework, indexing his concepts by means of be- 
havioral activities; but at the same time the terms "appetite" and 
“aversion” suggest subjective, internal, emotional phenomena. His use 
of the term “aversion,” despite its behavioral implications, can be 
equated without much trouble to fear (or perhaps fear and anger). 
But the term "appetence" is more ambiguous. It can, of course, be 
roughly equated to "hope"; but as we have seen, hope itself is am- 
biguous and bidimensional in that it implies both the relief and the 
excitement which are occasioned by stimuli previously associated with 
satisfying states of affairs. (A provisional resolution of this difficulty: 
has been proposed in Chapter 6.) 

We may conclude our consideration of Craig's article with a final 
paragraph in which he says: 


In the theoretically simplest case, which I think we may observe in 
doves to some extent, these states | of activity | bring about the appeted 
situation in a simple mechanical manner. The organism is disturbed, actively 
moving, in one situation, but quiet and inactive in another; hence it tends 
to move out of the first situation and to remain in the second, obeving 
essentially the same law as seen in the physical laboratory. when sand or 
ly copodium powder on a sounding board leaves the anti-nodes and comes 
to rest in the nodes (p. 92). 


It is interesting, in passing, to note the implied references to clec- 
tricity in both Freud's concept of cathexis (as a "charge") and in 
Craig’s concept of appetite. As indicated elsewhere (Mowrer, 1960, 
Chapter 7), the influence of electronic and "servo" devices upon con- 
temporary theory is also very marked. 


IV. The Work of L. T. Troland Re-examined 


In scanning the history of American psychology over the past half 
century, one’s eve is also caught by "Troland's concept of “retroflex 
action," which has interesting parallels with both field theory and two- 
factor learning theory. In a book published in 1928 and entitled The 
Fundamentals of Human Motivation, this author 


gives the gist of his 
theory in a chapter headed "Nociception, Beneception and. Retroflex 
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Action"; and it is this chapter, and more par 
with which we shall now be concerned. 
ng the carlier part of the chapter, Troland says: 


icularly the latter half, 


In summar 


Thus, the neutroceptors may be regarded generally as recording the 
environment, and the relation of the organism thereto; whereas the noci- 
and beneceptors t nsmit to the cortex constant reports concerning the 
effect of these factors upon the welfare of the individual or species (p. 215). 


One might expect this author then to equate neutroceptive stimuli to 
those which, through conditioning, become signals and to equate no- 
ciceptive and beneceptive stimuli to the so-called unconditioned 
stimuli. But this does not occur. Instead, Troland follows a different 
line of thought. He says: 


We must now consider more in detail how the cortex can be assisted 
by such “welfare reports.” are to be of any biological value, they 
must help in removing the organism from environmental relationships which 
are detrimental, and in establishing or maintaining such relationships as are 
L Now, thus far we have intended our classification of receptive 
mechanisms—as nociceptive, beneceptive or neutroceptive, respectively— 
to be of a pur ly formal biological character. In other words, we have i 
down no physiological criterion which would enable us to detect a noci- 
ceptive ora beneceptive system by a mere examination of the neural mecha- 
nisms. Their definition has been formulated in terms of the status of given 
nerve channels or activities with respect to the total biological situation. 
However, we have already stated. the principle that the action of noci- 
ceptors upon the nervous system is such as to inhibit cortical proce 
whereas that of beneceptors has a facilitative effect upon what the cortex 


benefic 


is doing (p. 215). 
We may now generalize this proposition in the following way. Noci- 


ception is accompanied by a decreasing of the conductances of operating 
cortical adjustors; whereas beneception is accompanied by an increasing 
of the conductance of operating cortical adjustors. We may assign to the 
process which is thus described the general name: retroflex action. This 
term is selected because the action in question is a kind of "back-kick" of 
organic effects into the cortex. The cortex, by its principle of trial and 
error, or random activity, initiates a certain line of response. This, in turn, 
produces certain actual or incipient organic changes which are reported 
back to the cortex via the beneceptive or nociceptive channels, and the ex- 
citations of these channels modify the cortical tendency. If the “report” 
is beneceptive or favorable, the tendency in question is enhanced, whereas 
if it is nociceptive or unfavorable, the tendency is reduced. These actions 

arded as being determined quite mechanistically, without reference 
to any accompanying pleasantness or unpleasantness, or any “intelligence” 
on the part of the cortical process. Facilitative retroflex action, based upon 
beneception, may be characterized as positive because it increases the given 
cortical conductance; while the nociceptive consequences may be charac- 


can be re: 


terized a negative (p. 216). 
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Psychologists of the 
stimuli always, or at le: 
“hedonic tone.’ 


introspective era had taken the position hat 
ast commonly, have what they referred to = 
` By this they meant, simply, that stimuli tend to ed 
inherently pleasant or unpleasant (scc Chapters 6 and 8). While = 
jecting the subjectivism of such concepts as pleasantness and unpleas- 
antness, Troland was still only one step removed from this tradition 


y . a at SO) dee as i Pur / harm- 
in that he argued that stimuli, if not "neutral," arc inberently ha 


3 PEIUS - 2 one at the 
ful or helpful to the organism In its struggle for existence and that 
principle of retroflex 


H y ake re- 
action operates in such a way as to make 
sponses which le 


3 K x h a ss which 
ad to nociceptive stimulation Jess, and responses W h 
lead to beneceptive stimulation more, lik 


In the hope of makin 
following illustrations: 


lv to recur. 
: : 2 ee vens iplo: 
g his theory more tangible, Troland gives t 


As examples of these two processes, we m 
of the "burned child" and the child with candy. In the first case we "m i 
suppose the child to be indulging in the form of random response bii 
is known as "play." One of the objects to which it responds is a candle: 
flame, and the re ction happens to consist in placing a finger in the ot 
As a consequence, the pain nerves are violently stimulated, and the corte 
adjustor process which is associating the afferent flame-pattern nerve Curr 
rent with the outward thrust of the finger, is abruptly reduced in conduc- 
tance. The consequence of this, in turn, is that the flame-pattern currents 
cease to pass through this particular junction, and pick out the next most 
conductive path, which will probably set off a sufficiently. different reaction 
to liberate the finger from the singeing effect of the flame. If it should hap- 
pen that the next most conducting adjustor does not bring about a sufficient 
change of posture, this adjustor will also be reduced in conductance in e 
same manner, the process continuing in this way until an escape is T 
Along with these Changes in the cortically controlled reactions there S 
also be a complex group of reflexes, which may in themselves assist in the 
withdrawal of the finger. The sympathetic nervous system will be aroused, 
the child will cry, ete, Wi 

Now the im 
cerned, lies no 
the cortic: 


ay consider the famil 


portant point about this 


3 w coii- 
c process, so far as learning is CO! 
tin the immediate e 


ffect of the nociceptive excitation upon 
j ally controlled behavior but in the fact that a permanent altera- 
tion has been made in the tendencies of cortical adjustment. The flame- 
finger-thrust adjustor has had its conductance decreased, and, hence, is per- 
manently less liable to recur. The next time the flame image is presented 
to the child's eyes, this particular path of response will be found sluggish. 
and some other reaction will probably: occur. Thus, physiological noci- 
ception, as we have defined it, has a lasting learning effect. In some cases, 
the original nociceptive operation will be inadequate to put the particular 
adjustor, which is concerned, our of action; but it follows from our general 
formula that successive Operations of this kind will have a cumulative in- 


fluence, cach decrease of conductance starting where the other one left 
off (pp. 217-218). 


HOPE, FEAR, AND FIELD THEORY 319 


In the case of the child and the candy, we may suppose him to respond 
by a movement which places the candy in his mouth. Thereupon the bene- 
ceptive sugar excitation is set off. This increases the conductance of the 
operative adjustor, which is responsible for the presence of the candy in 
the mouth. Consequently the candy remains, or is swallowed as a conse- 
quence of the deglutition reflex which is aroused; and thereafter, the child 
shows a greatly increased tendency to seek and eat sweetmeats. In this case 
there is positive learning, or a real enhancement of the conductance of given 
nerve paths over and above the original values. The enhancement will also 
be greater than would be effected by the principle of exercise. . . . The 
behavior of the individual under these circumstances should consist in a 
repetition of the movements, or a maintenance of the postures which bring 
about the beneceptive process (p. 219). 


Clearly, Troland was attempting to formulate a psychology of re- 
ward and punishment but one which would by-pass the difficulties 
which arise when these concepts are defined as involving, respectively, 
the subjective experiences of pleasantness and unpleasantness. Thorn- 
dike (1913) had tried to resolve this problem by operationally defining 
unpleasantness as an experience which the organism tries to avoid 
and pleasantness as an experience which the organism docs nothing to 
avoid or seeks to repeat. But there is an element of circularity here: 
in studying the effects of reward and punishment what one is en- 
deavoring to do, precisely, is to explain the fact of avoidance or ap- 
proach. 

More recent thinking has taken still another turn. Now the em- 
phasis is neither upon pleasantness or unpleasantness nor upon biologi- 
cal welfare or injury, but upon drive reduction and drive induction. 
In general, drive reduction is both pleasant and beneficial to the or- 
ganism and drive induction is unpleasant and actually or potentially 
harmful; and their use in defining the conditions of learning avoids 
any suggestion of circularity in that thev can be defined and identified, 
in most instances, without reference to the facts we are attempting 
to explain, notably approach and avoidance, facilitation and inhibition 

of behavior. Drive reduction, if shortly preceded or accompanied by 

a formerly neutr: al stimulus, can give to that stimulus the capacity to 
arouse hope (secondary reinforcement); and drive induction, if shortly 
preceded or accompanied by a formerly neutral stimulus, can give to 
that stimulus the capacity to arouse fear (secondary motivation). And 
these two emotions, as they become attached to both exteras and 
response-produced stimuli, "structure". (constitute) the individual's 
"total. psychological ficld," which is assumed, in revised two-factor 
theory, to control and guide overt behavior. 
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factor theory is that Finem by Edward Chase Tolman. Yet it is not 
casy to give an accurate and adequate summary of Tolman's views 
concerning learning. They have undergone change and repeated re- 
vision over a period of some thirty years, as new experimental facts 
and logical contingencies have emerged. Perhaps the main. difficulty 
is that Professor Tolman was “ahead of time" in that he had, from the 
beginning, high standards for what behavior theory ought to be and 
do, long before such theory was at all fully dev cloped. F rom the out- 
set Tolman has insisted that he is a Behaviorist" of sorts, but he has 
also expressed liv cly dissatisfactions with what has traditionally pen 


as Bebawmonsn. It would be an interesting task to start with his 192 

paper, “A New Formula for Beleiviorism. and follow down through 
the years his unremitting efforts to develop a theoretical system 
which would not have the limitations of other neater, but manifestly 
oversimplified, schemes. Instead, we shall try to pick from his writings 
a set of propositions that capture the spirit as well as substance of his 
endeavors and which will highlight both the similarities and differ- 
ences between his thought and the point of view presented in pre- 
ceding chapters. We shall attempt to do this in two stages: first a 
mere outline of his position and its relation to two-factor learning 
theory as presently formulated, then a somewhat more detailed and 


analytical presentation. 
Perhaps most central to and most characteristic of Tolmanian theory 
rning is sig» learning. Although behavior is al- 


is the view that all le 
tered as a consequence of learning, this is not because there has been 
a change in the synaptic resistance of neural pathways leading from 
particular stimuli to particular muscles, but because certain stimuli, 
or signs, have become integrated with, or into, certain "sign-gestalt 
expectations." The latter Tolman defines as a belief or conviction or 
“hypothesis” on the part of the learner that "such and such per- 
formances or behaviors (if carried out) would be successful in reach- 
ing such and such a goal" (1945. p. 237). "These expectancies funda- 
mentally are merely sets in the nervous system aroused by environ- 
mental stimuli” (p. 237). And, he adds, "goals" may be cither positive 
(desirable and to be sought) or negative (objectionable and to be 


avoided ). 


To sum up [says Tolman elsewhere], it appears that it is the actual 
leadings-on or not-leadings-on.—including such sub-varieties of this func- 
tion as “obstructing,” "being an alternate to.” “being a shorter way than,” 
or “a longer way than,” or “farther off than," and the like, which constitute 
the means-end relations; and it is the aroused preparatory sets in the organism 
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for such possible leadings-on, or not-leadings-on, and their various sub- 
varieties which constitute what I would call the means-end expectations 
(1933b, p. 402). 

In other words, Tolman has held that learning leads to knowledge, 
to cognitions, instead of specific, stereotyped performances, or “habits.” 
And with this statement of the essence of his position we are ready 
to compare it, at leas 
learning 
learning 


tina preliminary way, with revised two-factor 
theory. The latter agrees completely that learning is sign 
and that it does not fix or stereotype performance, It algo 
agrees that learning results in a change in “intervening variables.” 
rather than in altered resistance in stimulus-response connections. The 
difference between the two positions lies primarily, it seems, in the 


nature of the assumptions made concerning the intervening variables 


themselves. In both systems these can be characterized, at least loosely, 
as anticipations or expectancies. But for Tolman these phenomena are 
“purposive” and “cognitive,” whereas in two-factor theory they have 
a somewhat less mentalistic, more corporeal, more dynamic quality. 
One writer—E. R. Guthrie. (1952)—has observed, somewhat fi 
cetiously but not irrelevantly, that a Tolman-type rat, after learning, 
would never do anything about it. He would, this writer Pagea, 
be perpetually “lost in thought” at the choice point, unmotivated and 
unmoved by his altered cognitions. Thus, in Guthrie's 


own words: 
The form of association theory which Tolman has adopted does not use 
the association of stimulus with response, but the association of cues, which 
he first called “sign gestalts” and later called “field expectancies,” with 
mental event. The early parts of the maze (Tolman, 19334) act not 
for action, but as signs that the goal is to come, cues for the rat to expect 
the goal. By "expect? in this sense Tolman does not mean action or prep: 
tion, but a mental awareness of some kind. His "sign-gestalt" formula leaves 
quite untouched the problem of what a rat will do with a sign. or of how 
signs are translated into action. “These sign-gestalt expectations," he says 
(19334, p- 249), “I assumed would be to the effect that the lier parts of 
the discrimination apparatus would have become a sign or a set of Mann 
the rats that the encountering of the food-compartments was to be S SM E 
by running through this discrimination apparatus." Signs, in "Tolinan's m ec 
occasion in the rat realization, or cognition, or judgment, or ROS. 
abstraction, but they do not occasion action. In his concern w ith $ h ices 
on in the rats mind, Tolman has neglected to predict what the 
So far as the theory is concerned the rat is left buried in tho 
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tion is that when signs are followed by negative 
fects” for Thorndike, "significates" “for Tolman, “unconditioned 
stimuli” for Pavlov), they acquire the capacity: to arouse fear and 
that when they are followed by positive conse 
the Capacity to lessen fear, arouse hope, and 
fully understood, see Chapter 6) to 
ganism is internally conditioned, or 
to make certain behaviors 
probable. In this scheme, 
but also goes (behaves), 


consequences ("cf- 
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(Gn a manner not vet 
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VI. Tolman's Theory Made More Explicit and 
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For this purpose 
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M tO examine it more 
à paper entitled “Theories 
1934 as a chapter in Moss? Comparative 
an begins the statement of his own position 
a "variety of the field theory" (p. 392). He says: 
The present writer starts from the doctrine of Adams and Lewin, but 
asserts that, for the problem of learning, the important relations in the 
“psychobiological fields" are “sign relations.” “Learning” he would describe 
as the new formation, or the reformation, of "sign-gestalts" within the 


larger psychobiological fields. The theory: is therefore to be designated a 
"sign-gestalt" theory.7 


Sign gestalts are to be conceived 
nificate, and a bebavior-route leading 
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of his positior 
we turn to 
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“It may help the reader to think of the expression, sign-gestalt theory 


lows. With Lewin, Adams, and others, Tolman, in the Passage cited, \ 

that organisms “live, move, and have their being" in fields 

of stimuli, Gestalten)—hence the latter half of the Modifying expression “sign- 

gestalt.” But he was also saving, somew hat IN Opposition ioca Piet hn 
orthodox” field theorists, that the stimuli which comprise any given field, or 

situation, have positive or negative potency valence,” s fhe errs , 
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"if behaved to in such and such a w av^ (that is, the behavior-route), will 
lead to “rhat” (thar is, the significate ). A sign-gestalt sets the animal to 
expect" that when he actually behaves, “this” field-feature will lead, “by 
such and such a behavier-route,” to “that” ficld-feature. And learning. ac- 
cording to the theory, W ill consist in the making or the remaking of such 
expectation-sets under the hard tutelage of the outcomes experienced as 
the result of the actual behaviors that are tried (pp. 392-393). 


appreciate "Tolman's position, it is nec 


In order to understand and 
of high generality, one 


sary to realize that he was aiming ata theory 
that would reconcile and integrate the facts of conditioning, trial- 
and-error learning, and the so-called higher le: 
l “this new formula for beha 
formula for all psychology—a 
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also to the addict of imagery 
tionship between his theory 
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Pavlovian terms, conditioned $ 
Folman himself develops this relat 


ionship as follows: 


The sign-gestalt theory and conditioned reflexe: The sign-ges alt theory 
asserts that the conditioning of a reflex is the formation of a new sign- 
gestalt. It asserts that à conditioned reflex, S hen learned, is an acquired 
CXpectation-set on the part of the animal that the feature of the field cor 
sponding to the conditioned stimulus will lead, if the animal but waits, to 
tlie fearire deco held corresponding to the unconditioned stimulus (p. 393)- 
The sign-gestalt theory and sign Jearning. Vt is oby ious by now that w hat 
rest jv does is to reduce all learning to the pattern of sign 
1 as the prototype for all learning. In sign 
learning, one ficld-feature becomes the sign for another field-feature, and 
the response that appears as 3 result is appropriate to the temporal and other 
Characteristics of this sign re tionship. Such a description has been shown to 
hold for the conditioned reflex. And, obviously, it holds for the cases that 


we have specifically labelled sign learning: the dog's response to the sound 
of my approaching automobile, the “flutter” response of Wever's (1930) cat, 
and the gentle dropping-off response of Warner's (19322) rats (p. 396).> 
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two major varicties: hopes and fears, representing 
pations of good and bad events (signific. 
mere sign learning explain changes 


. respectively, antici- 
ates) to come, But how can 
in behavior of the kinds noted 
in trial-and-error learning? At least a provisional answer to this ques- 
tion has already been given, in an carlier chapter, for two-factor 


theorv. Let us now study Tolman's way of dealing with this prob- 
lem: 


Trial-and-error learning consists 


according to this |sign-gestalt| theory, 
in learning "what" each of the initially alternative responses "leads to." 
Thus, given the situation show n in | Fig. 9-1 |, the theory savs that what 
the animal learns is that the initi ond: 


1 al field situation V, responded to by the 
response R,, will lead to the resultant ficld-feature fi responded to by 
Ra, will lead to the field-feature fa; ete. The result of its learning is, thus, 


that F, becomes the sign for all the different alternative significates fut 
f4, and so on. Or, in other words, it learns the w hole array of alternative 
sign-gestalts, F,—R,—f,, F,—R.—f. and so on. But the animals. final 
taking of R, (the correct response) rather than any of the others is some- 
thing not part of this learning per se. It indicates the further fact that under 
the motivation conditions of the moment, the one significate f 
manded by the animal than are any of the other alte 
fy, fa, fy, and so on (pp. 396-397). 


5 is more de- 
rnative significates 


fi 
RU 
fo 
Ree 
T ae E Fig. 9-1. Schematization of trial-and-error learn- 
p CL» R—>f ing in terms of sign-gestalt theory Clolman, 
a HR EN 1934. p- 397). : 
Fee Bra f, 
Rs 
a 
5 


This schematization has some very interesting and 
tures. First, it will be noted that it differs from the usual fan-shaped 
diagram used to represent trial-and-error learning in that, instead of 
showing merely a hierarchy of S—R bonds, it attempts to indicate 
in addition, the effects, or consequences, that follow: any: aeta do. 
sponse. And here we see where the principle of sign learning. Or 
ditioning. may be said to be involved even in trial-and-error, selective- 
response learning. Although Tolman speaks of "sien. 
somewhat more inclusive w ay, it is cle: 


suggestive fea- 


con- 


Sn-Bestalts" in a 
ir, from the d 
text, that a given response Ri, for example, which 


consequence or significate fi, will become, in some sen 


tagram and the 
Is followed by 


SC, a sign of fi 
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Ordinarily, in conditioning theory, we think of a sign as a stimulus, 


Not as a response; but when we recall that virtually all responses are 
how the significance, or mean- 


stimulus-producing. then we can sec 
at may be called stizzulus 


ing, of f; can become connected, through wh 
(see Mowrer, 1960. Chapter 2). 


Mediation, vo the response Ry 
as saving that “a conditioned re- 


Tolman has already been quoted 
s acquired expectation-set on the part of the 
of the field corresponding to the condi- 
al but waits, to the feature of the 


flex, when learned, ts an 
animal and that the feature 


tioned stimulus will lead, if the anim 
conditioned stimulus" (p. 393)- In trial- 


and-error situations, the picture differs in one, and perhaps only one, 
important. respect. Here the animal docs not await for the significate 
or UnCS; instead the animal produces the UnCS, by its own behavior. 
Then, depending upon whether the UnCS is good or bad, the stimuli 
associated with this behavior take on the capacity to arouse, respec- 
ively, hope or fear, and the tene the response to recur will 


field corresponding to the un 


Jeney for 
decreased. 


be correspondingly increased or 
he notion that trial-and-error learn- 


In order to make more specific t 


Ing is, in the final analysis, dependent upon conditioning, let us take 


segment, OF sequence, from Tolman's diagram: 
i mä typically Vhorndikian depiction 
the Fa—Rı part of the sequence would 
would be implied (as the effect of the 
1 to involve strengthening of the 


a single “sign-gestalt” 
eg. F,—R;-f,. AS already note 


of trial-and-error learning, only 


he represented, although Ri-fi 
R); and learning would be assumec 
FIR, (SR) bond. Tolman makes a different assumption, as does 
revised two-factor theory: ‘Tolman, using a highly molar language, 
savs simply that a total sign-gestalt expectation, F,—R,-f,. is formed. 
Figure 9-2 shows a more molecular way of thinking about the prob- 
lem. Here, certain additional details. omitted in Fig. 9-1, are filled in. 


Fig. 9-2. A Tolman | "sign-gest I (top line) 
Fa is a drive ( Tol- 
which produces 


This produces 


made de 
nade more explicit (below). 
füan e 2 2 à 

Dan calls it a “field-feature 


Some instrument response Ri 


certain response-correlated stimuli Srey shortly Fa Ry fi 
followed by fi, an effect (or “field-feature ) M caet s 
A ) 1 a T cte 
Which produces some molar response R, of tSr ----.2q 
(hope or fear) E R 


whic : e 
Which (r) is the conditionable 


Component, According to the 
would expect subse- 


acilitated or in- 


nature of this 


Meaning™ reaction, one 
que 

]uent performance of R, to be f 
hibited, 
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response R, of which (r) 
on the basis of now abundant 


Significate or effect f, produces some total 
is the conditionable component. And 
evidence, it has already been posited that R, has certain immediate 
stimulus consequences, which are represented by S4. (response-cor- 
related stimuli). Therefore when the response-correlated stimuli, Sp 
are followed by the significate f, (unconditioned stimulus), which 
produces a total reaction R, we expect r to "associatively shift" from 
fi to S. The result is that if fi is of such a nature as to elicit an r 
reaction of fear, then the response which produces S, 
sequently, to be inhibited; wher 
duce an r reaction of hope (or appetite), then the res 
produces S,. will, on future occasions, be facilit 
this way, following leads provided by 
sistent with assumptions previously 
called trial-and-error learning 


will tend, sub- 
if fi is of such a nature as to pro- 


ponse which 
ated, strengthened. In 
Tolman and in a manner con- 
made in this book, we deduce so- 
from conditioning principles. Or, said 
differently, when we make "Tolman's sign-gestalt notion (F,—R;-f,) 
more explicit, we arrive at precisely the 
acquisition and inhibition as has been delineated in Chapter 2d 
At this point there is a temptation to digress and review 
paper in which Tolman (19332) approaches the same 
somewhat different angle; however, this paper will be refe 
more appropriately, in another connection ( Mowrer. 1 
Instead, let us conclude our synoptic review of Tolman's thinking 
with a final quotation from the document under 


discussion, Here, 
reference to “the sign-gestalt theory and the higher forms of 
ing," Tolman say 


same conception of habit 


another 
problem from a 
rred to, 
` 1960, Chapter 


in 
learn- 


The higher forms of learning were classed, at the 
chapter, as cases in which the final Correct response is controlled, at least 
in part, by temporally absent or by unique, Previously: unreacted-to fea- 
tures of the field. It now further appears, however, that we Must really dis- 
tinguish between two subclasses. First, there are the delayed re woe 
of problem and the double alternation type. In such problems, t 


beginning of this 


action type 


he situation 
"Some years ago Youtz (1938) showed that in respect to * 


A reinforcement, ex- 
tinction and spontaneous recovery,” both Pavlovian and trial-and-error l 
behave in much the same ways. And recently Miller (19594), in refe 
Youtz's work, has said: “These laws seem to be the same for classic 
tioned autonomic as well as for trial-and-error somatic response re 
either escape from electric shock or rew arding a hungry animal with food, 1 
Proponents of multiple factor theories of learning l see Chapter 3| can attracr 
more serious attention if they can rigorously specify differing laws Associated 
with supposedly different types of learning" (pe 261). If, as we now Suppose, 
conditioning is the primal form of. learning and if trial crivative 
thereof, rhe Youtz findings would follow as a m 


arning 
rring f 
ally condi- 
inforced by 


and error is a q 
atter of course. 
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I primarily one of temporal absence. And, secondly, there are the "reason- 
ing" and "insight" sorts of experiment, in which the situation is both one 
of temporal absence and one w here an important controlling. feature has 
result of the given behavior but has 


not, as such, been experienced as a 
to be "inferred" (pp. 399-400) 

Bur what, finally, are the new laws, if any, involved in such higher forms 
a learning? We do not know. We can merely sav thar these higher forms 
Certainly involve all the laws of mere trial-and-error learning plus some 
New ones. And the new laws, W hatever they may eventually prove to be, 
will be concerned with the ability to build up sign-gestalts thar, on the one 
hand, include in the sign field features that are temporally past and, on the 
other hand, include in the significate ficld-features that are Creatively inferred. 
x srv for building up these sign-gestalts? 
) attempt to. answer. We human 
s would 


qe symbolic processes. nec 
his a question that we shall not here 
beings know from our own experience that such symbolic proce 
undoubtedly be helpful. But whether the logic of the situation absolutely 


requires them, the sign-gestalt theory will not at present attempt to say 


(pp. 401-402 ). 


immediately preceding 


Some of the problems referred to in the 
re, with the aid of 


in later chapters. whe 


€ wei . " 
]uotations will be discussed i 
atisfactory solutions may be forth- 


rece * x 
ecent research findings, more 5 
ened £ 

oming (see also Mowrer, 1960). 
to indicate the wide range of problem 


Concerned for more than a quarter of a century E | 
a too simple and manifestly incom- 


Here we should, however, look at 
Tolman has directed attention 
and differences be- 


Bur these passages serve usefully 
s with w hich Tolman has been 
and to show his un- 


Willingness to be satisfied with 
ilate: pe : HNE 
plete ty pe of conceptual scheme. 
m to which 


O > 2 
ne more type of proble Iman, 
à the similarities 


5 Jul y a e e 
nd which will point up further 


and two-factor theory. 


twee : 
Ween sign-gestalt theory 


e 


VII. The Problem of So-called Place Learning 


versus Response Learning 
as given rise to 3 considerable literature. on 


l'olman's theorizing h : 
"as compared with “response learn- 


What is known as “place learning spon 
ing." Thorndike, Hull and others have held that learning is largely 


à matter of "habit formation" and that "habits consist of specific 


S—R bonds, which determine t 
tion (S) is present certain responses or 
cur. By contrast, Tolman has stressed cc 
edge of certain “means-end relationships," which are purposive and 
» not involve a stereotyped, fixed form of 


hat when a given situation or motiva- 
response patterns (R) will oc- 
»gnitions, knowledee—knowl- 


3! à ; 
goal-oriented, but which d 


behavior of any kind. Once a situation has been learned, i.c. "cog- 
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nitively mapped" (1948), then an organism, according to Tolman, can 
reach its goal by any of a variety of specific response routes, without 
prior rehearsal or practice. Having once learned 

object is located, the question of bow 


V to get there presents little or 
no problem, assuming, that is, that there are not insurmountable 
physical barriers. 


where a given goal 


Stimulus-response theories, while stated with different degrees of sophis- 
tication, imply that the organism is goaded along a path by internal and ex- 
ternal stimuli, learning the correct movement sequences so that they are 
released under appropriate conditions of drive and environmental stimula- 
tion. The alternative possibility is that the learner is following signs to a 
goal, is learning his way about, is follow ing a sort of m other 
is learning not movements but meanings. This is the contention of Tolman’s 
theory of sign learning. The organism learns sign-significate relations; it 
learns a behavior route, not à movement pattern. Many lear 
do not permit a clear distinction between ' 
is a single path with food at the end 
opportunity, there is no way 
stamped in by reinforcement « 
purposes and cognitions ( 


ap—in other words, 


ning situations 
these two possibilities, If there 
and the organism runs faster at each 
of telling whether his responses are being 
o whether he is guided by his immanent 
Hilgard, 1956, pp. 191-192). Í 
No attempt will be made here to review the m 
problem of place versus response Je 


any studies on this 
> arning which have now been pub- 
lished. In fact, we shall not even attempt to describe, in exact detail, 
any one of these studies But for our Purposes it will be useful to 
consider a simplified or, “idealized” version of a study 
reported in 1946 by Tolman, Richie, & K ish. i 

Let the reader Imagine an elevated open maze forming a perfect 


cross or X. With each arm thereof Measuring, let us sav, six feet 
in length. Now further imagine that the a 


zum four arms of the maze arc 
DRE: ini point in the cardinal directions of the compass and that a 
hungry rat is placed at the end of the Fast arm, y ith food (not visible 
at a distance) in a little hole at the end of the Nürth arm. After a few 
trials, the rat will, of course, 3 E. E 


f Y . i » 
go promptly and directly from the East 
arm to the North arm, and there obtain ) 


one may Sav, 


i dá pen and ear food. Clearly the 
rat has thus acquired a “habit, but we Cannot say whether this 
“habit” represents place learning or s : 


response learning, 


f However, the 
ollowing Manner, Suppose that, 


issue can be put to the test in the 
instead of being released onto the maze from the end of the E: 
the animal, after tr: ning of the kind described, 
end of the Hest arm. If the ‘habit Which has been acquired is a mat- 
ter of response learning, the rat will, as before, turn right at the choice 
point, which will take it out onto the South arm of the maze. But if 


st arm, 
is released from the 
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the "habit? is a matter of place learning, then, at the choice point, 
the animal will turn Jeff and enter, as usual, the North arm of the 
maze. 

On the assumption that, in a situation of the kind described, the rat's 
ne of hope to both placc-cor- 


initial learning. involves the conditionit 
reasonable. surmise would 


related. and response-correlated stimuli, a 
be ther when tested in the manner indicated the rat would show (by 
hesitation and vacillation) some conflict at the choice point bur might 
very well decide in favor of the place-correlated stimuli (see also 


Mow rer, 1960, Chapter 7). |f many rats we 
ning right at the choice 


re trained and tested as 


Indicated, some of them might persist in tur 
point (and thus moving spatially awav from the goal of food); but 


most of them would probably dct morc “sensibly and abandon the 


"correct? response in favor of the "correct". place. This, in point of 
fact, has been the common empirical finding (see, for example, 
Galanter, 1955; Waddell. Gans. Kempner, & Williams, 1955; but see 


also Scharlock, 1955. and Grice, 19482, and Bugelski, 1956. pp- 94 Rye 
and there has been some tendency to infer from such results “that 
the disposition to orient tow ard the 


goal is simpler and more primi- 
tive than the disposition to make right turns (Tolman ef al, 1946, 
p- 228), 


But, as is at once evident alter the situation so that 


one could easily 
Jominate. Suppose. for example, that the 
rather than open, thus excluding 
that place learning would be 
(sec Aderman, 19357. 1958). As 
“place-learning is simpler 


Fesponse learning would. prec 
X-maze were tubular (cunncl-like) 
all extra-maze stimuli, lt is now likely 
Much less stable than response learning 
Folman, Richie, & Kalish themselves observe, 
than response-learning Jonly | in situations. where there are marked 
CNtra-maze cues" (p- 229). 
psychology would seem to demand that 


A strict. stimulus-response : ? S 
is possible is response learning; 


the only form of learning 
Whereas a strict “field theory? à à l 
B place learning. But evr position suggests an intermediate interpreta- 
ton which squares quite nicely with all the facts: Normally, in so- 
called habit formation, hope is conditioned both to place- and to 
response-produced stimuli and whether. in case of conflict, one or the 
other will gain the ascendancy depends upon a number of considera- 
tions, including the relative int 
Campbell, i954; Restle. 19573 Robinson & McGill. 1958). 

As will now be clear. ‘Tolman and co-workers have been more 


Interested in and have tendec 


which 
* would perhaps insist that all learning 


ensity and numerousness of cach (cf. 


] to give greater theoretical importance 
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to place learning than to response learning; bur, inherently, sign-gestalt 


theory does not demand that conditioning occur only to environ- 


mental stimuli, any more than docs revised two-factor theory; con- 


ditioning can, and apparently. does, occur to proprioceptive and other 
internal, response-produced stimuli, as well as to environmental, place- 
produced stimuli. The former type of learning Was rather dramatically 
illustrated, a few years ago, in the following situation, A graduate 
(Harold Underwood) was teaching several rats to “shuttle,” from one 
compartment to another, by means of a mild electric shock delivered 
from the floor grills of the compartments. 
rat, on the first occasion when shock w 
ment, escaped from it by backing 
the other comp: 


By some coincidence, onc 
as presented in one compart- 
through the little door leading to 
artment. On the second trial, it so happened that the 
rat was again facing away from the door and, as before, backed 
through it. And on the third trial, although this time 
the door when the shock Was presented, the 


fully turned around and backed out of the 
the other one (cf. discussion of Guthrie's theory of learning, Chapter 
8). Only on the fourth trial did the rat get the “idea” that where he 
was going was more important than how he got there; this time he 
quickly ran through the doorway Thus we see that both 


head first. 
place- and response-produced stimuli, if associated with drive 
and cause an ore? 


facing toward 
rat nevertheless carc- 
one compartment and into 


reduc- 
anism, when sub- 
to reinstate them. 


r this issue of place versus re- 
mental rather than decremental 
ularly clear that it is not at all 
‘ho would seriously debate the 


tion, became positively: "cathexed" 
sequently in the same situation, to seek 

It is particularly instructive to conside 
sponse learning in the context of incre 
reinforcement. Here it becomes partic 
a question of cither-or but of both. W 
question of whether a rat, as a result of going to 
or three times and getting shocked there, 
place or the action that took him there? 
come conditioned both to the place 
and to the stimuli produced by the 
sequence. Having abandoned, 


a particular place two 
would thereafter avoid the 
Obviously, 


fear would be- 
Where the 


shock was experienced 
antecedent 
once and for 
and bond-stamping-out conception of habit 
question of whether fear gets connected, or connected more 
to stimuli associated with a response or with 
will almost certainly alter cases; 
both instances is the same, there 
whatever. Here again, then, we se 


response or response 
all, the bond-stamping-in 
One sees no point to the 
strongly, 
a place. Circumstances 
and since the principle involved 
is no theoretical problem 


€ how completely revised two-f, 
theory dissolves the apparent differences between so-called 
ment theory and so-called field theory. 


in 
at stake 
actor 
reinforce. 


HOPE, FEAR, AND FIELD THEORY 333 


In € wg : iki 
Chapter 8, attention has already been called. to. some striking 


similaritic " 
Hs p between the present approach anc 
“lel: : s : 

land, Atkinson, Clark, & Lowell (1953)- 
eve Td $ 1 i 

n more apparent in the following passage: 


1 that suggested by Mc- 
This pa allelism becomes 


avoidance must not be understood simply as 
a stimulus ina spatial sense. Thus “rage, 
response, even though it 
M ust be defined in terms 
REA ecto discontinue, remove, O! from a certain type of 
eae and not in ternis of its own characteristics. Attack has, as its 
(IS AM removal of the source of stimulation in the same sense that with- 
s d docs. Approach must also be defined functionally tc^ it is any ac- 
v, the objective of which is te maintain, or pursue a certain 


ki em E 
kind of stimulation (p. 35) 


- Fhe terms approach and 
going tow ards” or “away from” 
d ut goes over into attack, is t 

ES something. Avoidance m 


an “avoidance” 


oing tow ards” 


r escape t 


"ug 


continue, 


ultimately have an "avoidance" 
(eke Chapter 6). What is more 
elland et al. are saving, appar- 
before, that “approach” 
a particular stimulus) 
7 some stimulus 


M hether approach activities 
(abient ) motif need. not. be debated 
immediately to the point is that McCI 
ently in complete accord with w hat has gone 
(in the sense of producing er getting more of 
May involve either movement through space powari 
th which now desired stimula- 


obice 3 ` 
bject or mere bodily movement WI 
And the same for “avoidance —1t in- 


tion je s ; 
n ds inherently associated. j j 
: from sonmc stimulus ob- 


t through space away 
nt with w hich now undesired 


in this way, the issue 


Volye ^ 
à Ives either movemen 
Ject or the iibibition of bodily 


sti : Lr 
timulation is inherently asec 
arning has only 


- moveme 
iated. Conceived 
Of place-versus-responsc le practical interest, and no 
theorer; GEN D 
Coretical importance whatever. 


V ^ 
lll. Place versus Response Learning 


Further Analyzed 

In an earlier mimeographed version of this book, the preceding sec- 
tion of this chapter appeared very much as 1t does here, and to the 
Writer it seemed quite clearly and fully developed. But as a result of 
classroom use and discussion, ne apparent that this material 


it bec 
had many implications which the w riter had not previously explored. 
Moreover, the students seemed to attach to it special importance and 
to be unwilling to leave it until their perplexities and interests were 
fully: &ntisfied. rom this reaction to the preceding section, the follow- 
Ine additional considerations emerged. 

ing one of the points made in the preced- 


Let us begin by paraphra: 
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ing section, Ordinarily, in a controlled experimental environment, the 
"right" response takes the organism to the 
words, response learning and place le: 
becomes evident in the "Tolman-Richi 


X-maze, conditions can be alte 
Now, in the test situation, the correct response (a right turn at the 
choice point) takes the animal to the w rong place; and in order t 
get to the right place (end of the North arm), the animal has to make 
the "wrong" response (ic. has to turn left, rather than right, at the 
choice point). Or, put otherwise, the À 

whereas the response cues say and the individual experiences 
a conflict which will be resolved one way or 
upon whether the place or y 
the Wickens experiment of 


“right” place. In other 
arning coincide. But now, as 
c-Ixalish experiment with the 
red so that this is no longer the case. 


) 


place cues now say “do this 
"do that,” 


the other, depending 
re predominant. (cf. 
1938, cited in Chapter 1; also Gibson & 
Mowrer, 1938, as discussed in Mow rer, 1960). 

But what this analysis overlooks is that in the 
“wrong” response at the choice 
still be right, correct, Suppose that on tri 
from the end of the West maze arm, 


the end of the South ar conditions, the animal went 
to the end of the North arm of the maze, not only would the re 
be wrong; so also would be the 


this kind, it should be but 


the response cue 


test. situation, the 
point (i.c, a turn to the right) might 
als when the rat is started 


food is now, in point of fact, at 
m. If, under these 


sponse 
place. And by continued training of 
a short time until the animal would always 
and unhesitatingly go right at the choicc point, regardless of the 
on which the animal was introduced to the mi 
so to say, an instance of "pure" response le: 
have an instance of "pure" pl; 
example, the food were aly 


arm 
rc. Here we would have, 
ning; just as we would 
jc learning if the animal found that, for 


ays in the North arm, regardless of where 


it Was put onto the maze, 

Here, perhaps more clearly than before, we see th 
is no "opposition" between place and response learning Whatever, Both 
are possible, and cach may be uniquely: useful, depending 
circumstances. Fortunately, we have an over-all] conce 


at in theory there 


upon the 
PUon of the na- 
“phenotypically” dit? 
forms of learning can be equally well derived. “Genory pically." 
are the same, i.c., both are dependent upon the Conditioning 1 
(or fears) to place-produced and or response-produced stimu 


ture of learning from which these two Fernie 
they 
of hopes 
; li. 

At the risk of perhaps claborating the obvious, let us look again at 
the rat which first backed from one Compartment to the other 


: i of the 
shuttle apparatus and then abandoned this behavior and started ry 


inning 


Nainly 


forward. Superficially, it looks as if this was a situation calling ı 
for place learning; and we are inclined to be amused by the rat's 
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ason that he was making 
He was “assuming” that 


behavior 3 4 : X 
eee on the first few trials, for the t 
why ; . : 
it might seem to be a rather silly mistake. 
ipparatus 


hows SF A " : 
© he got to the other side of the was important, when, in 


dO MNA: SR 5 
point of fact, it made not the least. ditference—just 50 long as he got 
by all odds, a superior 


there f t ři 
re. And since moving forward is normally, 
backwards, our lowly 


form of locomotion as compared with moving 
subject seemed pretty ludicrous. 

But as Tolman himself has reminded us (see \owrer, 1960, Chapter 2) 
We must nor fie tae “lofts” or hasty in such situations. Here, for all 
sible that the nature of the locomotion, 
And we, as experimenters, could 
1 turn the shock 


Me rat knew, it was entirely p 
ij x ell as its direction, W as important. 
easily. have made it so bv simply deciding that we woulc 
OF when the rat “shuttled” only if he did it backwards. If this “rule” 
à of this particular rat would have 
at all. In other words, response 
c stupid than place 
and 


ha y ` » S 
d been in force, then the behavior 


heen highly etficient- cand not funny 
| ferior to or mor 
upon the circumstance 
have good potentialities for 


DNE 

ar : à 
Ning is by no means inherently in 
re seen, 


enis À 
a TN. 
? inning: it all depends, as we hav 
i ; , Nr 
s fortunate indeed that living © 
both, s 


reanisms 


alled to. the fact that. place 
with decremental or incre- 
punishment. We may now 
learning corresponds 


attention. is € 


li " 3 : 
V the preceding section 
cither 


and re s 
! response learning muy occur 
th reward or 
response 


me rece ; 
ental reinforcement, iC. WI 


note. | "T N . 
te, in addition, that in the latter cas" 


termed inhibition or passive avoidance be- 


he hat we have previously > 
Mor: and that the corresponding place learning. would be, surely, 
dance behavior Aes the subject, if put in a place 
1, would be strongly instigated to 
is the picture equally clear 
Clear. ves, but also 


x vies of active avol 
e € it had previously bee 
uu reinforcement 


n shockec 
must now ask. 
is concerned? 
f such reinforcement, an organism 
this, manifestly. is response learn- 
regardless of how 


3 Nc decremental ! 
; complicated. If, asa result o 
Comes to make a particular response, 
ing. And if he regularly goes tO a particular place, 
he gets there, this. equally clearly. is place learning. But we cannot, it 
Seems, here make a distinction bet active and passive approach 
yv we have four 


een 
ad it possible and convenient to 
avoidance behavior. Instead it 
Seems that the counterpart avoidance behavior is W hat is 
learning and that the counterpart of active 
rmed. place learning." Later, in 


behay R 
li havior, in the same wa 
Istinguish between active and passive 
| of passive 
Yere ter 
€ termed response 
avoid; k 2 : ES 
vidance behavior is what 15 here te 
of these four situations can be made somewhat more ap- 
ries of movements, an < 
the organism, if ar a distance will subse- 


Mig 
inn SERE A 

as follows: If, after a 5€ 
punished. 


anism arrives at a par- 


ticular ; 
ar place and is there 
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discrimination learning, we shall find 
ification docs not quite exhaust the logical and, 
y al possibilities; but for 
(cf. also the quotation from McClell 
the preceding section). 

Here it need only be added th 
of this section were quite right. 


connection with our analvsis of 
that this four-fold cl 
indeed, psychologi 


present purposes it will suffice 
and et al. appearing at the end of 


at the students referred to at the outsct 
This matter of the distinction between 
place learning and response learning is indeed important, It is, so to sav, 
the "battleground" between stimulus-response 
theory and ficld theory; and if 
that revised two-factor learning. theory handles 


attendant issues, then one has a firm and insightful hold upon that 
theory itself, 


bond ("reinforcement") 
one sees, clearly and fully, how it is 


and resolves the 


IX. Behavior Variabi ity in a 
More or Less "Open" Field 


Psychologists, when studying the so-called trial-and-error behavior 
of animals under laboratory conditions, have usually employed cither 
a maze or a problem box in which there is only one "route" or be- 
havior sequence leading to the goal. Occasionally: they have, how ever, 
employed situations involving two (or more) equally good wavs of 
reaching the goal. Suppose that on the first trial, in thc simple "maze 
shown in Fig. 9-3, a rat (or other laboratory animal) takes the right- 
hand pathway from thc starting box to the goal I Š 
presumably, that the animal had, to begin with 
to take this pathway than to take the 
gets reward for having done so. 


0X. “This Means, 
> 8 stronger tendency 
other one; and, n ' 
One might expect, therefore, that the 
animal would immediately fixate on this pathway and never try the 
alternative, left-hand pathw ay. But this is not what happens, Despite 
reinforcement for having taken the initially preferred pathway, the 
rat alternates irregularly from one route to the other." 


moreover, he 


quently refrain. from returning thereto (passive avoidance) 
placed in the setting where the punishment previously: 
will strive to escape therefrom (active avoidance 


and if 
occurred, the 


). If, on the other h 
a series of movements, an organism arrives at a particular. place 


rewarded, the organism, if at a distance will subsequently 
lace and if passively placed in the setting where the rew 
will tend to stay there. 


Passively 
organism 


and, after 
and 
Strive to g 
ard previously 


is there 
et to this 


Occurred, 


"For a somewhat different set of facts and a quite different way 
ing them, see Guthrie & Horton (1946). These investigators, report 
ble degree of stereotypy in cats in a situation involving alternative 


ing a given problem. 


of interpret. 


3 Considera. 
Means of soly_ 
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advanced the suggestion that behavior 
ght arise from what Hull 
(1943) Was to term "reactive inhibition." The basic notion here was 
and etfort produces fatigue, and 
1 though a behavior 


Mowrer & Jones (1943) have 
variabili J : : E n 
Iriability of the Kind just described. mi 


om Every activity. involves effort, 
atigue is at least mildly punishing; therefore, ever 
sequence is consistently. rew arding, it is also somew hat punishing; so 


that one might expect the stimuli associated with that response to take 


G 
Fi . " ; vu 
9. 9-3. A simple maze with equally satisfactory al- 
ternati A 3 
ternative routes to the same goal. The starting box 
Is Me, : hr 
labeled S, and the goal box G. 
S 


t—"good™ because the response 
it involves work (see Chapter 
ally, would prompt 
s inferred that, 


on ambivalent meanings for the subjec 
leads to a desired gra "bad" because j 
F1. Pisarad involve conflict. and, conjectura 
the organism to vary its ac 1 from this 1t V 


ther things equal, the greater ture of effort involved in 
greater would be the tendency toward 


periments failed to confirm 


tivities. Ane 
the expendi 


reaching a given goal the 
response variability. However. several ex 
this inference. i QAI : 
EEN hypothesized that response variability, at least in a maze 
Situation, might come from the “anticipatory 
© as) which, on previous runs, have been strongly 


the goal). 


intrusion,” carly in the 


maze 
laze, of responses (tur! 


rewarded (because nearer 
est of this not 


A specially designed H-maze 
see d ion: but the results were agai 
seemed to offer a good t ion; t Its were again 


B 
Negative (Mowrer, 19508, pp: '7" 17 3). , 
Pretty clearly, then in a situation involving more than one equally 
(or even unequally —see Whiting & Mowrer, 1943) satisfactory routes 


factors operating other than those which, 


to 
Ww. are goal-relevant. Or perhaps 


à given goal, there arc sd 
from the experimenters point of vie 
We have not as vct grasped some ve 
and learning. Of course. the fact that an organism has one drive or 


H "TU 
notive, such as hunger. 


ry basic principles of motivation 


does not preclude the. possibility. that it may 
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simultaneously have other motives: 
competing motives would act as 


as “distractions” and, hence, could 
account for deviations from that type of behavior which might seen 
again to the experimenter 


+ to be most efficient and business-like (see 
A x he s 
Bijou, 1958). Hull (1943). on the other hand, has approached thi 
problem along the following lines: 


fear, fleas, sex, and so forth. These 


At this point we must intr 


oduce a factor to be taken up in detail a 
little later—that of the 


spontancous oscillation or va ability in habit strength. 
It will be sufficient here to say only that there is reason to believe that the 
effective strength of all habits when functioning as reaction potentials Is 
subject to continuous uncorrelated interferences, presumably mainly from 


processes arising spontaneously within the nervous system, and that the 
magnitudes of these disturbances are distributed approximately according 
to the "normal" law of probability. On the assumption that the oscillations 
in habit strength are largely uncorrelated, it follows that the weaker of two 
competing habit tendencies will frequently be depressed only slightly when 
the stronger chances to be depressed relatively much, with the result that 
the weaker habit will dominate on that occasion (pp. 14971350). 


Whether the 


above is merely ad boc 
nings of a re 


al explanation is Not casy to sav, In any event, this ap 
proach to the problem 


has not been ge 
Psychologists have continue 
solutions. 


neurologizing or the begin- 


nerally accepted; and other 
d to search for more strictly pay chological 

Some experiments, lar 
at the University of M 
perplexing matter, 
N. Dember in a se 
follows: 


gely unpublished, which have been carried out 
ichigan promise to shed new light upon this 
These have been briefly reviewed by Dr. William 
minar report, given in 1954, which reads in part as 


The general problem | 
of choice, It Concerns the 
faced with two or 
motivation which 
with respect to the 
choice to the ne 


am going to discuss is, 


essentially, the problem 
prediction of what an 


organism will do w hen 
More alternatives. This is rcally the. basic question. of 
alternative will the organism take? Is there some order 
alternatives it will choose? Can you predict from one 
xt? IlI be forced to talk about overt, observab 
Motor response, but contained in what Tm saving will be 
that at least some of it refers to cognitive events—the 
and percepts which are not direetly observable. 


le choice, i.e. 
implicit the notion 
sequence of ideas 


After referring to the "reactive inhibition" theory of behavior 
variability and to certain related. experiments (cf, Glanzer, 1953; 
Montgomery, 19512, 1954). Dember gocs on to say: 

There is 


a set of experiments by Dr. C. R. 
which I tho 


Brown of this department 
ught vou should all become fami 


ar with. Dr. 


Brown was in- 
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of rats in simple, unstructured situ: 
| just about fit into 
of this table—and 


eer T the exploratory behavior i 
E Fa pre an enormous round table w hich would 
Dr. Br ne The rat finds himself sitting in the middle 1 
: "nds up by the ceiling looking down and recording what the 
rat does. And it turns out to be “some of the most consistent behavior ever 
observed. 
Suppose the rat makes a little excursion to the east. It is predictable with 
almost perfect certainty that the rat will return to v here he started from: 
he walks away from the place w here he first found himself, he will re- 
turn there, Thar is one of the main principles that comes out of Dr. Brown's 
Work. | See references in Chapter 6 to exploratory behavior and its probable 
relation to fear. | j 
oe does the rat do next? | 
M Piu that is dn the opposite 

s second trip he'll venture 


another excursion, this time to 
direction. from his previous excur sion. 
a little farther than he did on the first. 

And again he'll return. The rat makes another excursion; this time 1 don't 
direction, except thar it will vot be east 
rth on the third trip. On the fourth he'll 
travel south, From rat to rat and day to day this pattern shows up perfectly. 
Now let us return to the original problem. How do vou modify what 
Seems to be such a basic patrern of behavior? What can vou do to the 
environment that will change this nice sequence of east-west, north-south? 


Uhis Dr. Brown has also answered. 

f Suppose vou put a little wooden box somewhere on the table. And the 
rat is again placed in the middle of the table. At first you will get the pattern 
Previously described, if the box is far enough from the rat so inae he can't 
See it clearly. AVG, however, in the course of his excursions the rat comes 
"pon the box, the pattern of his behavior is markedly changed. The rat 
Spends most al his time sniffing around this object. If chere 1 another object 
Somewhere else in the field, he'll spend a lot of time exploring one and then 
Move over to the other, pl 1 with that for a while and. return to 
the first one. In short, what the rat now does is to orient his behavior around 
these objects. Aei it seems that the 


presence of differential complexity 

in one portion of the environment disrupts the basic pattern of. alternation. 

Vhis is the principle thar Robert Earl and 1 were interested in when we 

built the TURON zc. One of the things We wanted to find out was w hether 
Wee M teen 1 Se 
ve could manipulate the visual environmen 


t of the rat with respect to this 
Variable of complexity in such a way as to modify the general alternation 
f pattern, We didn't want CO use objects as Dr 


Brown has done, because 

this might lead to problems of interpretation. We searched for visual pat- 
terns that would work as his objects had (pp- 9-11). 
t of the research just described. (with the 
lished by Dember, Earl, & Paradise (1956) 
and Within-Stimulus. Alternation; the 
lus Complexity." which begins: 


He makes 


^ 


= 


nk it Š : 
à ink it's possible to predict the 
"Y we 3 

west, Let's sav the rat goes nf 


irounc 


s More recently, an accoun 
Infinity maze") has been pub 
1 B " 1 I n0 
H a paper entitled, “Bet cen- 
Response to Differential Sumu 

sb) has shown that under certain conditions normal 


K rechevsky (19374 1937 : yes 
Tats tend to select the more variable of two paths to similar goals. Butler 
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(1953), Montgomery (1954), and Myer: and Miller (1954) have uem 
strated that animals will perform specific ac s if the acts are. instrun E 
in providing the animals with interesting environments. — TT: 

The present experiments, on the empirical level, fit well into suc os ie 
text. Abstractly, they derive from the authors’ concern with the paren ar 
problem of alternation and the more general problem of choice behavior, 
Both of the experiments presented in this article involve the following pro- 
cedure. Naive, “satiated” rats are given access over extended periods of 
time to two contiguous circular pathways. The animals can move freely 


within and between paths. The walls of one circle are lined with a simple 
pattern. The pattern in the adj 


acent circle is relatively more complex. T hat 
aspect of choice-behavior which is of primary concérn here is the relative 
amount of time spent responding 


to each stimulus Cpe 19s 


And the authors state, in summary: 


In both experiments the records we 
units. Data were presented for only 
yield substantially the same results. 
conformed to prediction [that mor 
complex portion of a “maze” 


re analyzed using three arbitrary time 
one of these analyses, since all three 
In both experiments the results closely 
€ time would be spent in the visually 
then in the visually simple portion] (p. 13). 


What is one to make of findings such 


bear upon the particular type of thinking 
this volume? 


In Chapter 1 (cf. also Sears et dl, 1957) a tentative distinction was 
made between "learning theory" and "behavior theory," the latter 
being presumably more inclusive than the former, Perhaps, in the 
studies just cited, the question is less one of what is learned than 
given a particular set of circumstances, of what will an organism do? 
The answer to the latter question presupposes a knowledge of what 
has been learned, but something more: a know ledge of prevailing 
environmental conditions, Still, we try to be objective in our \ 
of learning, i to use performance criteria of learn- 
are back at the point of having to be 
havior in order to t; 


as these? And how do they 


which is being explored in 


it is, 


study 
€, We are inclined 
ing; and with this we 
predict and explain be, 
ing. 

Certainly, in the objective stud 
mary, metabolically: pr 


able to 
alk Meaningfully about learn- 


y of le 
oduced drives into 
acknowledge the secondary drives that ar 


of stimulation. But perhaps we h 
variables which we have noted 
make predictions. 


arning, we have to t 
account and we 
€ cued off by v 
ave been, even SO, toc 
and on the basis of 
As already suggested, the mere fact tl 
a college student) is hungry does not, forthwith, 
being “interested” in things other than food. A 


ake pri- 
also have to 
arious forms 
too narrow jn the 
which we seel, 


K to 
aat a rat (or 
prevent him 


from 
nd if Interests—j 


€n an 
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organism’ - : 

à ganism's hopes and fears—vary in a somewhat uncontrollable 
anner, it f i St i idi 
ner, it follows that behavior will also have considerable fluidity. 
which one ideally secks in 


Such: 4: 44 MOX. 
h a view lacks the svstematic rigor 
igorous" than 


science Sap. . 
nee, but we do not want a theory which is more 
go from highly to 


the reality to which it applies. Perhaps. as we 
degree of predictability necessarily 
y be that behavior re- 
cviously neglected 


less highly structured situations, the 
drops, Or, as the Dember work suggests, it ma 
Mains predictable but that we must take other, pr 
l complexity. Eventually, as we move 


variables i i 
lables into account: e.g. vista 
or “open,” situations, the 


fro 

3 m highly structured. to unstructurcd, 
umber of variables one has to take into account becomes unmanage- 
1 breaks down, and all one can do is 


isc predictior 
Yr probabilities (scc Mowrer, 1960, 


ably: large and pre 
3 Speak in terms of tendencies € 
Chapters 8 and 9). 

It is possible that, even in such controlled envi 
provided by mazes, some of the so-called "errors" made by an organ- 
ism are, from its point of view, not errors at all but instead represent 
behavior relevant to motives and goals which, for the moment, are 

the organism, though nonexistent for 
7 all monomaniacs; but, in 
gs “on their minds” at 


ronments as those 


Een at importance to 
Xperimenter. Ideally, rats 1n 
pretig, they probably have a nui 
any given time, and it ‘need not sur 
onp this multiplicity of motivation. 

Elsewhere (Mow rer, 1960, Chapter 
the main body of revised two-factor learning theory 
to the problem of response sclection and “choice”; but, in pa: 
reference should be made to a parenthetical comment appearing in the 


X `r . 1 it: 
Paper by Dember previously cited, to WIE 


a maze are 
nber of thin 
prise us if their behavior sometimes 
is made to relate 
more specifically 


ing, 


6) an attempt 


and response is not a uscful one; what 
present context, Is response-feedback, 
om any other source of stimu- 


n stimulus 
t in the n 
al properties fr 


X ^ he distinction betwee 
svi Mean by response, at l 
oe is no different in form 
atio " 

n (p. 5). 


This statement agrees well with one of the basic assumptions involved 
direction in which the 


in revi : ue 
I revised two-factor theory, P 
integration of habit theory and field theory can most felicitously move. 


As Dember implies, à response is most accurately and meaningfully 
defined in terms of the stimulation it produces; and our assumption is 


reinforced, 


and suggests the 


that responses are positively or negatively, and subse- 
€ 

]uently repeated or 
DsSV c 1 i 
Psychologically speaking. 


on precisely the same basis. Thus, 


restrained 
stimuli; but obviously, there 


responses are 
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can be stimuli other than those which a € response-produced. The 
total "field," psychologically speaking, therefore consists. of both 
extrinsic and response-correlated stimuli and their 


attendant mean- 
ings; and these meanings arc being const 


antly collated, compared, and 


ific actions and inactions, But more 
detailed reference to this process should be reserved for 


organized into "choices," i.c., spe 


a later discus- 
sion of the role of consciousness (Mowrer, 1960, Chapter 7). 
X. Hull on the Problem of “Behavior in Relation 
to Objects in Space" 
As I see it, the moment one express 
various potentialities of behavior 
of one or more variables, he h 


es in any very general manner the 
as dependent upon the simultaneous status 
as the substance of what is currently called 
field theory. My habit-family hypothesis is presumably a field. principle 
in this sense. It is probable that my equation expressing the goal gradient 
hypothesis (gradient of reinforcement) and even the one expressing the 


generalization gradient (gradient of irradiation of Pavlov) might also qualify 
as bits of field theory, 


and, some might object to this usage 
on the ground ch; ature are not peculiar to gravitational 
or electromagnetic fields but are almost universal in natural science theory. 
Moreover the Variables and the constants, together with their inter-relation- 
ships, which have so far appeared in equatior s expressing behavior theorie 
seem to differ radically from those which appear in the equations of physica 
field theories. The notable lack of such equations in current Gestalt field 
formulations, while paradoxical, is probably symptomatic, For such reasons 
I prefer to be rather sparing in my use of the expression “field theor yr. df 


one has the substance, the use of the term is scientifically: unimportant how- 
ever valuable it m 


ay be for purposes of propaganda, Bur if one means by 
field theory what I had indicated, I am all for it and | sce no inherent dis- 
agreement on this point between stimulus-response theory and Gestalt 
theory (Hull, 1943, p- 287). å 


Finally, a word about Hull's remark 
field theorists, as 1, are 


On the other h 
t equations of this n 


on the poor way in which such 


handling field theory, ] Suppose that Mr. Hull will 
understand that even the cry "Let's take field theory away from the field 
theorists” would be sweet music for the | Š : 


atter, Lam even optimistic enough 


to believe that, in time, Hull and I Vill agree as to the main ch cteristics 


of field theory, in particular that this isa Method of analyzing causal rela 
tions and constructing theories and concepts rather than a “theory” in the 
usual sense of the term. In the meantime what counts is that after a decade 


ake ie a d 
of refutation, psychology is turning more and More toward field theory: 
(Lewin, 1943, p. 290). : 


If he could have lived until 1952, Lewin would have had in 
tion of seeing his prophecy fulfilled; for in that year 


book containing a chapter with the title appearing 


satisfac- 
Hull published a 
ithe heading of the 
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to articulate his own systematic 
i which ficld theory has been 
beginning of this 
[but] students 


Vea in iid Here he attempted t 
Pifticutah UM of the problems with 
chapter, ae ghee As he points out in the 
at sts * PST. must necessarily occur in space |) 
Mergen ij^ for the most part not explicitly recognized approach 

ance behavior as a division ot psychology requiring special 
Instead, thev have been largely 
an organism comes to make a 
r situation. Here the emphasis 
C specific muscular 


e (pe 215) 
akie n with the process whereby 
is upon ae eer to a specifie stimulus © I 

rat Murrav (1935) has called “actones, 
i as opposed to acts, which are defined 


contracti 
actions and movements, 
a particular g 


More | 2 : 1 i 3 
in terms of the attainment of yal than in terms of 


the 

xact behavioral elements involved.!? 

Hull makes exte! 
and svstematizes them in 


In hi : f i 
V his approach to this problem. aded use of Holt’s 


on "adience" and "abience." but defines 
n wav, Here it is not possible tc 
its complexity 
which give th 
gist of his owt 


a complete summary of 


) give 
but there are a 


and length; 
c historical contest of the 
a mature thinking about 


Mull s 
u 3 : 
fev x Chapter, because of 
pie 
prol WA Ar near the end 
blem ; : 
ip. em as Tull saw it and the 
th » obvious in animal behavior that 
Fwd cannot. be overlooked. Adience ! lelv employed by animal 
: "s Chologist as an indicator of the results of learning in the greatest variety 
if wear, 28 i [TE o 
ft Situations. Unfortunately this has been done with little or no explicit 
Phe i Hat of the inherent complexities involved in the process itself. It 
ga dclieved that this is the reason for some of the theoretical confusion re- 
gardi sts S 
rding maze learning. 


Che Brets "Mm 
€ first important publicatior 
Lewin in 1933- 


and abience are S 


The adi 
le facts of adience i 
yas been Wile 


iin the ficld of behavior of organisms to- 
An amplification of substantially: 
;a book in 1935: These works pre ented 
f the gen | field, and raised at a qualita- 
concerning behavior toward objects 
ion of subsequent workers, even 


war : 
he a alae in space was by 
an BRE ^ material was published å 
tive | “cedingly valuable analy sis © 
in & ene a large number of problems col 
space which have occupied the attent 


can psychologists, 


sition on the part of Ame 


al disp‘ : 
onse-wise rather than “spatially,” 


to think resp 


m 
AS opposed to the gener 
oodworth: "Since neither chain reflex 


duri 
Se ng the past several de 
Brie DN aa . 
Day cite a remark made m 
Nor 


des, 
1938 by W 
he rats be 
The most 
ibing the rats concrete behavior, though 


-ounts for t havior in the maze, we ask once more 
learns. 
1 descr 
thar the rat 


obvious answer, which has been 


Ned pattern. accot 
Eel b that the animal Be 
ated peatedly by investigators ir De 
in their theories. 15 simply learns the. place. By place we 

1 containing objects in s tial relations" ip. ias). It is 
lates both forms of learning with 


mean a " 2 

lie a concrete situatiot 
ype 

M ped that we now have a theory 
qual felic 


that accommod 
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though Lewin himself seemed not to 
spatial problems as such. : f d , p mm 

In 1938 the present writer published à manuscript written in 1934. S ee 
attempted to apply a quantitative mathematical analysis to ROTC. ed a. 
problems, in particular to those involving the goal gradient haan *» 2s 
Since the manuscript was already written. on the. basis of the by-d jen: 
abandoned logarithmic formulation of the goal gradient, this form of the 
hypothesi appears in the published study. This article gave what is believed 
to be the first quantitative mathematical derivation of the problem of adient- 
abient equilibrium. It also gave quantitative analvses of several forms of ae 
barrier problem. The author's present view is that these latter analyses are 
defective in that the principle of afferent stimulus interaction and stimulus 
patterning was not employed (Hull, 1943, pp. 349 ff.). 


Around the year 1940, Neal E. 
Brown and several others, 
experimental attack 


have been much interested in the 


Miller, in association with Judson S. 
began an exceedingly sagacious and ingenious 
on this series of problems, employing albino rats as 
subjects. Fortunately. as early 1942 Brown published in detail a part of 
this experimental wor together with the important germinal idea that 


the goal gradient principle is not the only factor operating in open space. He 
says: 


"It can be shown, however, th 
with the concept of the spatia 
(Brown, 1942, p. 209). 


ata number of these facts are 


also in accord 
al generalization of 


conditioned. responses" 
In the opinion of the present writer, the princi 
the most important single advance recently 
reader has already seen, it has been exploite 
going chapter. While much of the 


ple just quoted constitutes 
made in this field. As the 
d on a large scale in the forc- 


work of Miller and his associates had 
not been published, owing to the participation of both Miller and Brown 


in the war effort, Miller was able in 1944 to include a summary of much 
of it in his chapter, "Experimental Studies in Conflict," which appeared in 
Hunt's Personality and the Behavior Disorders (Miller, 1944, PP- 431 ff.). 
Miller's theoretical analysis is essentially behavioristic in nature and while 


technically qualitative in form, clearly advances the subject to a new high 
level. The experimental results are admirably: quantitative (Hull, 1952, pp. 
270-272). 


From this last quotation we see that 
Was in some way the most outst: 
was moving toward common 
believed that this confluence 
thought is faciltated 1 
further unification 
confidently expect 


> in his final work, Hull, who 
anding habit theorist of his 
ground with the 
of two great stre 
y revised two-factor le 
and Strengthening of this 
ed in the ye 


generation, 
field theorists, It is 
ams of psychological 
arning theory and that 
arca of psychology. can be 
ars immediately. ahead. 


] 


Gradients of Reinforcement 
(Type-D and Type-1) 


and Temporal Integration 


At has long been known that learning occurs most readily when a 
Stimulus or response (which produces stimuli) 1$ followed immediately 


by reward or punishment; but it is also well established that a rcinforc- 
NG state of affairs (rype-D or type-D can be somewhat delayed and 
still have an effect. These facts have given rise to the notion of a 
declining gradient m the effectiveness of a reinforcement as 105 oc- 
ar ved in time from the stimulus or re- 
10-71). The purpose of the 


non in the context of re- 


Currence is more and more remo 
sponse with which it is associated (Fig. 
Present chapter is to examine this phenome i : 
vised two-factor learning theory- And because a stimulus that is 
Spatially far or near with respect to 
temporally remote or proximate, the 
à hc discussion, 


an organism is likely also to be 
topic of reinforcement gradients 
follow :4 the preceding chapter of fic 

lows naturally upon t in the p g chapter, ld 
theory, i 


l. The Question of Backward Conditioning 


Before proceeding with the main business of this chapter, it will be 


i è RS 
Iscful to consider a side Issue: 


namely. the question of whether con- 
in the forward direction or can also occur 


ditioni 
Pritts can occur only 
'ackwards." The Polish neurophys 
345 


iologist. J. Konorski (1950). has 
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treated this problem in a particularly 


trenchant manner. He begins 
with this definition of conditioning: 


Fig. 10-1. Learning as a (hy- 
I ] | [ 


po pothetical) function of the in- 
t terval between the occurrence 
3 € of a stimulus S tor a zesponse 
5 R which produces stimuli) 
E =. and the occurrence of rein 
E forcement (decremental or in- 
= cremental)., The resulting 

curve is commonly called the 

Delay of reinforcement h 


gradient of reinforcement. 


If two stimulus 5; and s, are applied in overlapping sequence, the stimulus 
Sı being antecedent, then, with repetition of such combination, a plastic 
change in the nervous system is formed, consisting in the 
quiring the ability to elicit the response of the c 


stimulus s, ac- 
the stimulus s, (p. 413) 


same kind as [that made to] 


Then Konorski sa 


NSS 


In our definition we have 
detailed examination, 


The first is the assumption that the 
termined by the sequence of the 
stimulus is alw avs conditioned 


introduced two statements which ne 


ed a more 


“direction” of conditioning is de- 


stimuli applied, ii, that the antecedent 
and the subsequent stimulus is conditionin n 
This assumption is based on the ast experimental evidence collected both 


in Russia (cf. Konorski, 1948, p. 19) and in America (cf, Hilgard & Marquis, 
1940, P. 174). It goes to show that if a "neutral" stimulus is preceded by 
an unconditioned stimulus the conditioned reflex to the neutral. stimulus 
cither fails to be established at all or is insignificant and ex anescent. It has 
been claimed that “backw. ard conditioning” (as it is called) cay be suc- 
cessfully elaborated (Schnirman, 1925; Switzer, 1930; Nezdanova, 1940 ), 
but as the experiments yielding the negative result are abundant and un- 
equivocal while those yielding the Positive result contain some Possibilities 


of error, we deem that the hypothesis Concerning the sequence of stimuli 
in the process of conditioning is well substantiated (P. 414). 


of evidence for 
predominantly: Cf 
direction has been ady 
troductory chapter 


Quite a different type believing that conditioning 
is normally and not exclusively ) in the forward 
anced by both Pavlov and Hull, In the in- 
of his famous be Conditioned reflexes, 


(1927) savs that one can, in f equally ue signal refle 
in support of this basic 


ok, 
act, speak 
notion he 


Pavlov 
XCs5; and 
says: 

Let us return now to the 


simplest reflex from 
started. If food or some 


s which our inve 
rejectable substance 


stigations 
finds its way 


into the mouth, 
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ciues yoke of saliva is produced. The purpose of this secretion is in the 
RA p to alter it chemically. in the case of a rejectable substance to 
the T S ash it out ot the mouth. This is an € sample of a reflex due to 
Sune: sica and chemical properties of a substance when it comes into 
tact with the mucuous membrane of the mouth and tongue. But, in 
addition to this, a similar retlex retion is cvoked when these substances 
are placed at a distance from the dog and the receptor organs affected are 
only those of smell and sight. Even the vessel from which the food has 
been given is sufficient to evoke an alimentary reflex complete in, all its 
details; and, further, the secretion may be provoked even by the sight of 
the person who brought the vessel, or by the sound of his footsteps. All 
these innumerable stimuli falling upon the several fincly discriminating dis- 
tance receptors lose their power for ever as soon as the hemispheres are 
taken from the animal, and those only which have à direct effect on mouth 
and tongue still retain their power. The great advantage to the organism of 
^ capacity to react to the former stimuli is evident, for it is in virtue of 
their action that food finding is way into the mouth immediately en- 
Counters plenty of moistening sali ind rejectable substances often nocuous 
to the mucuous membrane, find a laver of protective saliv a already. in the 
Mouth which rapidly dilutes and washes them out. Even greater 15 their 
Mportance when they evoke the motor component of the complex reflex 
of nutrition, i.c, when they act ? stimuli to the reflex of secking food. 
Here is another example—the reflex of self-defense. The strong car- 
Divorous animal preys on weaker animals, and these if they waited to de- 
fend themselves until the teeth of the foe were in their flesh would speedily 
be exterminated. The case takes 0n a different aspect v hen the. defense re- 
flex is called into play by the sights and sounds of the enemy s approach. 
Vhen the prey has a chance to save itself by hiding or by flight. 

How ean we deseribe, in general, this difference In ithe dynanuc balance 
Of life between the normal and the decorticated animal? What is the general 
mechanism and law of this distinction? Ir is pretty evident that under natural 
Conditions the normal animal must respond not only to stimuli which them- 
Selves bring immediate benefit or harm, but also to other physt al or chemical 


light. # ad the like—which in themselves only 
stimuli; though it 15 


not the sight and sound 
y itself harmful to the 


pele ia aves of sound, 
of tie approach of these s 
d veast of prey W hich i5 11 
ze and claws (pp- 13714)- 

| Thus we see that the fundamental 
Nemispheres is that of reacting to SI" 
of interchangeable significance (p 15)« 

of course, 
responses. He took no cog- 


smaller animal, but its 


and most general function of the 


als presented by innumerable stimuli 


What Pavlov did 70t scc, was the motivational and 
of conditioned 
of the fact. as we now 
d that signals of safety and si 
erful inner forces then largely determine 
Pavlov, along with the other 


mediati NES 
lediational significance D : P 
£ perceive it, that signals of 


nizanc 

AN whatever facti 

danger r i eh 
dange PARE pre Mi isfaction arouse 
2 > á 
hope and that these two pov 


and direct external, objective behavior. 
reflexologists completel ese phenomena and, in so doing. 


v by-passed th 
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prevented his theorizing and ex 
generality and power which tl 
Pavlov saw some thir 


perimentation from having the maximal 
dey might otherwise have attained. But 
nes with great clarity, and among these was Gers 
tainly the fact that conditioning, in order to be biologically mcaning- 
ful and useful, must be a forward- 
process. 
Hull, similarly, in a paper which was published. in 
title "A Functional Interpretation of the : 
the “biological utility" which arises, “blindly but beautifully” from 
the circumstance that living organisms, throu 
conditioning, can react to signals ad 
rence of the event signified. Stimuli Which come after 
event (either “positive”—helpful, or “ne 


cannot function as signals of such 
be no re 


D "ri y 
rather than. backward-occurt ing 


1929 with the 
Conditioned Reflex.” stressed 


gh the mechanism of 
aptively, in advance of the occur- 


a significant 
gative"—harmful) obviously 
an event; and there would therefore 


should become c; 


ason why such stimuli apable of cliciting 


any part of the response made to the event itself. On the other 
has been abundantly demonstrated, a stimulus which char 
occurs in advance of a significant event c 
becomes connected to Some part of the reaction called 
event itself. Hence the theoretical assumption and the empirical 
evidence that conditioning isa forward, rather than backward, process. 
Therefore, in the present chapter, in speaking of gradients of rein- 
forcement, we shall Consistently assume that we are speaking of varia- 
tion in learning as a function of the time interval between some stimu 
or response and an ensuing reinforcement, decre 


"Thorndike (1933), Muenzinger et al. (1936), and a few 
entertained the idea, on the basis of certain r 
of reinforcement, at least as far 
This possibility is discussed 
ditioning and Learning 
ported this notion 
Wise 


hand, as 
acteristically 
an be extremely useful if it 


out by the 


lus 
mental or incremental! 


other writers have 
esearch findings, 
as reward is co 
and evaluated in 
(1940). Because | 
+ it will not be further 
find no consideration here of Hull's 
called "goal-gradient" 
has shown that it is 


that the gradient 
ncerned, may be “bidirectional,” 
Hilgard & Marquis’ volume, Con- 
ater research have nor sup- 

The reader will like- 
postulations concerning the so- 
d by Hilgard & Marquis. Later work 
rather than primary Phenomenon and of less 
Y supposed, A somewhat related notion. will, 


apter 12. For further e that. condition 
only in the forward direction, sce Perkins (1955), who sa 


tion is strongly supported by recent work (Borasio, 
1952; Mowrer & Aiken, 1954; Spooner & Kellogg, 
indicates that effective conditioning occurs only yw 
Instances of apparent backward or strictly si 
be explained as special cases of Pseudocondiric 


and theory 
considered here, 1 
carly 
(1932), also. discusse v 
à derivative 
importance than was formerl how- 
ever, be discussed in Ch Vidence ing occurs 
“This interpreta- 
1952; Fitzwater & Reisman, 
19497) which rather cle 
when the 
simultaneous 
ning. H 


"s: 


arly 
CS precedes the US. 
conditioning 


can all 


RIDE Pseudoconditioning. in turn, 
appears to be only a special instance of the Operation of stimulus generaliza- 
tion (Wickens & Wickens, 1942) Óp- 345). And Decese 


(1952) cites Spooner 
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I. 
Mee in Reinforcement Type-D, 
ntrinsic Stimuli 


of ie on ^ ip ed reinforcement may be expected, on the basis 
lt ilie ieee "i learning theory, to fall into four major categorics. 
E P ace, they may be classified whether they 

à cremental reinforcement. Or increment 
in the second place. they may be cross- 
is or stimuli which the rein- 
allv produced) or intrinsic 
more experiments of 


according to 
al reinforcement, Les 


rewar 

mos pies and, c 

PRTA ae tow hether the stimult 

(response a ows I8 CNETIDSIC (environment 

dio ER Ir so happens that one or c 

Fesults zh; a four types have been reported in the literature, with 

the bag ar cantor very acceptably to W hat one would expect on 

"e s of theory. 

the Ba s of delay -of-reit 

gest and largest history 


which happens to have 
lecremental reinforce- 
roduced stimuli, 1C. the so-called delayed- 
be confused with the delayed- 
scc. Mowrer, 1960, Chapter 6). It will 
that ‘Thorndike, in his 1913 formulation 
d that reward and punish- 
antecedent be- 


forcement study 
Ment of merinsi is that involving ¢ 
reward het eig responsc-p 
response ype ot paperinen (not tO 
TO reet a ot experiments 
of the l i , from Chapter noe 
Ment SER of r eet. explicitly 

are effective in proportion to 


hav 
‘vior. Ee said: 


recognize 
their proximity to 


akening effect of 


ingness (OT the we 
f the connection 


he closeness © 


The 7 
TR strengthening effect of satist 
etwi Ingness) upon a bond varies with t 
ee 1 
en it and the bond (1913+ P- +) 
ade earlier, The 
the phenomenon, 


yrndike (1898) had made at 


In fac 

[o act, more than a dec 
ast passing reference to as follows: 

sclection, from among 


ere consists of the 


a Ls Ba proce of learning h dus Leu te OF tram, c 
And yt : ota certain impulse to act incon s a certain situation. 
of SR first business Is tO discover the a M dE i ction. The result 
impul ; discovery Was dogmatically EP n 3 bs pre is lecture. “Any 
xem se to act which, m à given pera eads m easurable feelings, 
v to be connected more firmly W ith that situation; . « + 1 say tends 
'ecause the pleasurable feelings must follow the act W ith certain limits of 


DING go (p. 71) 
i; gd. 


ion "that the percentage of backward condi 
training” (p. 82), thus sug- 
ion. The only writer w ho 


e observat 
high. bu 
ie to some sort 
estioned 


& Welles 
: Kellogg GOo47) for th 


tione 
d responses is initially t decreases durir 


gesting 4 
m g that it is an artifact du 
as, withi : 
ithin recent years. seriously qu 


| this view is Razran (1956a). 
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E: T ; "dé at 
er page 3s d i c remarked th: 

And, on a later page of the same study, Thorndike r S 

“the pleasurable feeling may not be 


contemporaneous with the act, 
but may come considerably later 


. and the act itself and impulse theret 
may not be in the lcast pleasurable” (pp 


Although the term, "gradient. of reinforcement,” was apparently 
first used by Miller & Miles in 1935, 


denoted by this expression h 


79-80). 


it is thus evident that the facts 
recognized in technical psycho- 
logical literature since the turn of the century; and one would suppose 
that they had been known, on the 

for thousands of years, 
1916, John B. Watson, 
Association, published a paper in which he ¢ 
this phenomenon on theoretical grounds and 
experimental observations tending to 
extensively reported elsewher 
summarized, in the Psy 


ad been 


basis of common observation, 
It is therefore surprising to discover that, in 
then president of the American Psychological 
]uestioned the reality of 
Stranger still, reported 
support his skepticism. More 
(Watson, 1917) 
cbological Bulletin (W. 


g this experiment was 


atson, 1916a), as follows: 
The purpose of this 


which govern the st 


experiment y to throw 
amping in of suce sful 
out of unsuccessful movements, According 
pleasure from a successful act st 
hold that the stamping in of 
of recency and frequency 
unpleasantness resulting 
A small experiment: 
of animals (rats), closely similar in age 
chosen. The problem box used was 
animal had to learn to ser 
an inch and a half in di 


through an opening to 
box w 


light upon the factors 
movements and the stamping 
to Thorndike the resulting 
amps in the reaction, Other investigators 
à successful act depends upon the principles 


and is not dependent upon the pleasantness or 
from the activity. 


al beginning was made upon this problem. 


Two groups 
docility, body weight, CU, were 
a modification of the say dust box, The 
atch under a bank of sawdust, find a small hole 
ameter, crawl under the saw dust box, and mount 
a raised floor. Screw ed to the floor of the problem 


à small metal food. dish covered by a perforated metal disc. The 
metal dise could be opened from the outside. The experiment was 


ducted as follows: The animal was put into the restraining cage and 
to work until the problem was solved. With one the 1 
raised immediately: upon the entrance of the 
allowed to feed until his appetite w 
metal dise was not raised until thirty 
the problem box. During those thirty si 
very active, biting and tearing at the 
cage, etc. At the end of thirty: seconds the 
animals allowed to appease their hunger, 

The learning curves for the 
identical (p. 77). 


con- 
allowed 
group the metal dise was 
animal and the anin 
as appeased. In the other 
Seconds after the 
*conds the anim; 
Metal food dish 


al was 
group the 
animal had entered 
ils in this group were 
Tunning around the 
metal disc was raised and the 


two groups of animals were almost ex; 


Anv student of modern beh 


avior theory can 
Watson failed to obtain a diffe 


at once suggest why 
rence in the le; 


arning of the immediate- 
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üm abus ok ecd i he experiment just described: 
pa nd ) feeding delayed primary reinforcement, but it did not 

v secondary reinforcement (cf. Chapters 4 and 5). Sight of the 
odors from the food itself could 
“bridged the gap” be- 
and the primary 
1, somewhat later. 


reward ; 
d and delaved-reward. groups int 


d er at er pes 
y have mediated the latter effect and. thus 
digging behavior 
] when consumece 
fore this explan 


twee a E x 
kt P. the instrumentally effective 
‘inforceme sat, d 
Hi "cement provided by the fooc 

Avever. i $ 
N OVER EET A good many years be 
coming, ; ry 


ation was forth- 


performed an experiment 


In the ; r » 
he meantime, in 1927. V arden & Haas 
r results. These 


some É yoa * f a 
what similar to that of Watson and obtained simila 


Writer i Kg a ; 
ers stated. the objective of their investigation by asking: 


; a maze habit, proceed 
immediately follows the 
i sries of in- 


tiga: fixation of a pattern response, Rs a 
Facies ca speedily hen the ic nae polit i 
tervals dt fe i ipi Mie e ey x Kee "The Pent study is a pre- 
liina e RSCAIpOD the rate of learning? | he R icai p^ 
: port of a sy stematic Iv estigation of this genera problem (p. 108). 
tson used, Warden & Haas 
end of which was located a 
A small metal food dish 
inverted funnel, heavily 
nded above the dish so that, by means 
the dish and then, after a time, 


c funnel permitted normal 


Inste: n ; 
us P of the problem box which Wa 
S RB NE (cight-unit) U-maze, at the 
i8. inches square. 


fo 

o / 7 

d box approximately 
of this box; and an 


ve ee o the fon 
le S ted with lead, w: 
lir x cord, it could be let down M, 

ted. Say the author “perforations 1^ th 


olfac es à 5 EZ 
actory stimuli during the period of delay" (P n). 
ith one group of rats the funnel was lifted, 


Ir : ; 
à 1 this experiment, WI 
lus giving access to the food, as soon as the food box was entered. 
a delay of one minute; and with an- 


Wi 
Ith a second group there was y 
of five minutes. All three groups were closely 


cc criteria © 


suspe 


other 
E Yer group, a delay 
SOI Pee : ae 
à nparable in terms of thr 
an $ PN 

d time.” Say the authors: 


f maze mastery: “trials, errors, 


erval between the pattern response 


" We must conclude that 2 s-minute int ] 
9 be learned and the feeding activity has no disadvantageous effect upon the 


fixation SF aie aire fabis in fact. the value for both trials and errors are 
cb HYS instances—slightly lower than those of the control [no delay | 
GoM although this difference in favor of the DUNE gnoup ian mm cs E 
Significant. . . . The average total time spent within the maze was, how- 

ever, somewhat greater for the s-minute prop. xs (p. 112). 
TRI Ecc roiie ST be considered as so much evidence against the 
latter insists. that the value of the "s tisfving 


law 
st 4 of effect, insofar as the „insists | 
state” is a function of its “nearness in time" to the act, or series of acts to be 


fixated (p. 116). 
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Even more striking are the results of an c 
known to Warden & Haas, which h 
1924. For 10 days prior to the 
particular cage. The anim 
required to run a maze as 


Xperiment, evidently nar 
ad been reported by Simmons in 
experiment proper, rats were fed in a 
als were then divided into two groups and 
a means of getting to this cage. One group 
received food immediately upon getting into the cage or goal box; the 
other group had to wait for food one and one-hal 
long a delay as this, there was no clear difference in the maze per- 
formance of the delayed- and the immediate-feeding groups. 

In 1929, Hamilton (née Haas) again attacked the problem of de- 
layed reward but with procedures which were rather 
those employed in the preceding studies, 
effect of delayed reward (food) upon th l 
lightly charged grill. Once having crossed the grill, the rats, instead of 
being admitted immediately into the cating compartment and being 
delayed there, were delayed in a sort of “anteroom.” Under these 
circumstances, it was found that a delay-of-reward of 
seconds was sufficient to reduce the willingness of r; 
trified barrier; and the effect was even more m 
animals which were delayed three minutes, 
before being allowed to enter the 

In a second study, H 


f hours. Even with so 


different from 
First she made a study of the 
€ willingness of rats to cross a 


as little as 75 
ats to cross the elec- 
arked with a group of 
after Crossing. the grill, 
food compartment, 


amilton required her subjects to learn to 


run 
a maze, with the same arrangement. for delaving reward: Pe. the 
animals, after getting through the 


maze, were cither adi 
artment and fed or 
' for one minute, three, 
delay was over, 


nitted im- 


mediately to the food comp were delayed, in a 


“restraining compartment,’ 
utes. “After the period of 
enter the food box + asin the control 
learned to run the maze, but the no-delay 
There was, how ever, no very clear diffe 
groups, though all were clearly. inferior 

Mrs. Hamilton has the following to sa 
difference between her own findi 
carried out by her and Warden: 


five, or seven min- 
animal was permitted to 
group" (p. 175). All groups 
group learned it most re 
rence between the 
to the no-delay 
N by way f 
ad those of the 


the 


adily, 
four delay 
group. j 
of explaining the 


ngs ar study previously 


The previous study differed from the present one 
spect: the animal was delayed in the same comp 
‘That is, in the former experiment the animal y 
presence of the unobtainable food, Whereas jr 
stimulation of the food could occur only afte 
the delay compartment into the food box. 
matter of place of delay was much bette 


in one important. re- 
AMUIPE «ath ple. food. 
as directly stimul 
n the present case the direct 
r the animal was released from 
In the Present experiment the 
T controlled—the delay compart- 


ated by the 
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Ment wy eris ^ EX z 
Bee qiie. separated. from the food compartment. Evidently the 
the latter be ang delayed in the same place W ith the food, even though 

e inaccessible, has little or no effect on the fixation process (p- 195)- 


Dy e 

Waly i ENSE $ i & 

> y of interpretation, Hamilton continues: 

findings in any very im- 


o relate our 
v Thorndike. . - 


kta 
ould seem to be impossible t 
formulated by j 
inciple that 


DOT 

' Det m S. the law of etlect as F 
the rim, ary to the law of effect, Thorndike develops the pr 
response dnd n one bond is derermined by the temporal closeness of the 
is Thenndilee ie satisfying of annoying state W hich follows. If we assume, 
then our ves hide to; that the incentive (food) produces satisfying state, 
“Upon the y x S would suggest that very definite restriction must be placed 
any TELA i. of the temporal factor in this type of learning. For, while 
al, within the limits investigated, cuts down very markedly the 
the longer intervals were hardly ‘more 
"nor afford us a basis for con- 
ishment" or merely 
negative sense, nor 
processes 


eHiciene 

effective a the learning process: 

lecture as ine the shorter. Our results da Dor an’ 

AS diver a W hether delay operated asa positive pun 

do they orn from the incentive situation m a purely n 

underly; er any clues as to the physiological (or neurological) 
ying the behavior called out CpP* 19 


DS possible to 5 


7-198). 
A npo knowledge. it is ne cc why there was 
r no differentiation in the leart various. delayed- 
the Hamilton 
tment was 
ompartmen 
reinforcem 
althoug 


h it into th 
action was experienced 


ng of the 
maze study: the stimulation 
associated, each time a sub- 
t. with primary reinforce- 
ent potency. Therefore, 
h not so satisfactory as 


Fils PRA used. in 
Jeet E Ne the delay compar 
tins ps "s ased into the food € 
siiis LUCA acquired secondary 

g into the delay compartiment, 
ectly throug 
ling; and th 
the delay 


eae ed to pass dir c food compartment, 
immedi SINUN hat reware 
in Meurs upon entering me 
limits i the same degree. regardless o 

ised) that the animal was retaine 


use 

us of a delay box represents an advance 

d by Watson, by Simmons. and by V arden & Haas, 1t still did not 
à actorv Way of demonstrating the diminishing 


delayed reward? This problem was, 


is satisf 
compartment and apparently 
the length of time (within the 
d therein. Thus, although the 
over the types of procedure 


Provide : icf. 
ge ide an entirely satisf 
ctiveness of progressively 
failed to reveal a gradient 

in 


of s Simmons study, P cited. apparently 

the DE reinforcement following reason. 
ceding compartiment during the delay period, the fact that the compartment 
preliminary training) with eating 


considerable secondary reinforcement 


reviously 


Although no food was 


ha 

Md been previously associated 
1 from the 
] entry 
ate primary rew 


outset 
into it to be about as effective in rewarding 


ard. Cf. the Saltzman study discussed 


seem 

pot s to have given it. 
eney g T 

maze ds and to have cause 

in C carning as was jmmedia 
chapter 4 
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however, largely solved by a different procedure employed in a study 
reported by Roberts, in 1930. 
The apparatus used in the 
Hungry rats were released 
m ; ^w 

at the right into the pr 


Roberts study 


" individually, from the starting box shown 
oblem box" (center 
gaining access to the feeding È 
ing the small strip of wood whi 


is shown in Fig. ro-2- 


). The correct response for 
ompartment, at the left, was touch- 
ch c: 


in be scen suspended by a string 


um 
| | f | 
| r1 l'I— 
D | 
JU | : 
| 
u 
i20 4 
SS 
Fig. 10-2. Sketch of Apparatus used by Roberts (1930, P- 36) in one of the first 
experiments. which adequately contro! led. secondary 


the gradient of primary rei 


reinforcement i 
ing box, the one at the 


[ n studying 
nforcement, The compartment at the right is the start- 
left the goal box, 


from the ceiling of the main Compartment, 
to the group, were used as subjects, 
touching the piece of wood, Were given immediate 
compartment, while in the other three 
5, 10, and 30 seconds, respectively, 
measured in all groups by r 
between release 


Four Sroups of rats, 


AM 10 
The rats in 


upon 
access to the food 
SToups, there was à delay of 
l Learning, ON successive trial, 

recording With a Stopwatch the 
from the starting box and the oce 
rect, or “key,” response. The results are 
the author says: 


one group, 


als, was 
ime elapsing 
time clapsing 
“rence of the cor- 
shown in Fi 1073. Of them 


g 
g. 


The “learning curves? for the diffe 
of the different periods of delay. 


rent groups showed di 


distinctly the eff. 
ere s ) ect 
The longer the delay, the slower was 
"learning" and the less prompt and stable the behavior Pattern at the on 
clusion of the experiment. 
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lt st 
vould 
SAT G e noted that a. variation of procedure led to results quite at 
A ` 
inean ( pe those published by Watson in 1917 and by Warden & Haas 
» 3 1 
I 5). | Because of its recency, the Hamilton study was evide ently 


no 
t known to Robert] 


REINFORCEMENT AN 


o construct a reinforcement 


Is his findings as clearly 
becomes progressiv ely 
ence of a specific 


El Roberts makes no attempt t 
Bipsordg i basis of his results. he regard 
an horndike's assumption that reward 

ive as it is remov ed in time from the occurre 


instru ` 
mental act. Why e arlier inv estigators, using differe 
a gri adient was not entirely clear to 


shows a good, intuitive 


ent procedures, 


had 
€ 
Rol found no evidence for such 
S 5 
m but in the following 
der 
standing of the concept of seconc 


remarks he 
lary reinforcement. 


| stage must in itself be a “rew ard” for the 


Skinner's later (1938) concept of 
close to the f food box and sub- 


as in the Watson experiment] was a very large 
ation. In anthropomorphic terms, the animals 
only puzzled by the apparently irrational 


jal action, cacl 
iry [compare 


in éhe chamber, 


bate such 

ces preceding : acti 

lect to oli pesa Me ne 

Part of E eR stimulation | 

knew w her consummatory situ 
re the food was and were 


140 

130 30-min. delay = 
120 d 
110 
100 


20 
Immediate 
10 release 


Me 
dian time to make correct response, in seconds 


ji- a- 3- 4- G- 4- B- 92- S- S- 6l- 


p- n- 25- 28- 
x » Q5 4 5 9 5 8 8 


Ex dede 
IS: AE 
Trials by groups of 3 


| the efficiency of learning as a function of 


differences ir 
and the occurrence of primary rein- 


Fi 
9. 10-3. Curves showing 
relevant act 


the 5 
fc time interval between the 
n 

cement (Roberts, 193% P gon. 


356 LEARNING THEORY AND BEHAVIOR 


manner in which they were restrai 


ned from feeding as soon as they arrivec 
at it (p. 50). 


While the experimental 


procedure employed by Roberts representec 
an advance over earlier 


procedures, it was in one respect not ideal: 
it was conducive, in the delay groups, to a certain amount of ANE 
learning," which accounts for the “hump” in the carly part of the 


learning curves for these groups. Because contact with the wooden 


bob or pendulum was probably not the last, or most “recent,” response 
which a delayed animal would have made before being admitted to the 
food compartment, there was a tendency. for these more recent but 
irrelevant responses to be reinforced more powerfully than the rele E 
response. These responses would then take precedence over the right 
response and would have to be extinguished, one 


sistent reinforcement of the corr 


after another, until 


ect response would 


During the two decades following 
other studies were reported which tri 


laved-reward learning by still more refined and exact techniques. It 
will not be possible here to review or even cite a good many of these. 
Suffice it to say that in 1943 Perin, working under the direction of 
Clark Hull, published a study, in which he empirically derived the 
gradient shown in Fig. 10-4. And a few years later Perkins (1947) 


carried out a different sort of study from which he 
forcement gr 10-5. 
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unreasonable to hypothesize that there is no primary gradient of reinforce- 
ment, but that ali learning involving delay of the primary reward results 
from the action of immediate secondary reinforcement which develops in 
the situation. Factors affecting the development of this secondary reinforce- 
ment would then be considered as responsible for all 
delay of reward. Such a hypothesis, it will be 
sity of explaining how the rew 
fluence something 


lients found with 
observed, climinates the neces- 
ard seemingly acts backw and eater to Ins 


which occurred earlier (p. 7) 


And, in summary, Spence says: 


The interpretation that learning under conditions of 
reward involves a backward action of the 
stimulus-response event is rejected. The h 
native to this conception is that all such learning occurs as the result of the 
development of secondary: reinforcement, the action of which is conceived 
to take place immediately upon the occurrence of the response. A prominent 
aspect of this theory is the concept of differential secondary reinforcement 
based either on cucs from the external environment or inner cues such as 
proprioceptive stimulation provides (pp. 7-8). 
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and logic just reviewed, Hull, in ins 
x took the position that the gradient of aedi 
reinforcement probably does not exceed, at best, five seconds jud tha 
it declines according to the function shown in Fig. 10-7, But ther 
a number of reasons for thinking that this 
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reinforcement are easily deriv: 


able. Let us represent the situation, with 
maximal generality, in the mar 


ier suggested in Fig. 10-8. Let S, repres 
sent any originally neutral stimulus or stimulus complex. Such an S, 
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` k f he bar 
the animal makes a bar-pressing response (R) to the smell of food on t Rey! 
a al makes a ba sing res es of fo s; 
a delay of 4” elapses before it receives the food reward (K^). Simultane d 
with the animal's reception of the food the goal reaction of cating (r,;) oC ids 
i pri i i ti s trace le -the apparatus 
and becomes primarily reinforced to the stimulus trace lefr by the app 
and response stimuli (p. 127). 
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su dual circuits of the kind shown: and since these circuits presumably go out 
Cessively rather than simultaneously» can deduce the gradual decay" typi- 


cally 
Y observed in a stimulus trace. 


one 


4g tO discover that the concept of 
Pavlov (1927) to account for the 
vhen there is a brief interval be- 


Jy occurs N 
S. But Hull though for somewhat 


rberating neural 
phenomenon 
-<-) for the notion that “After the 


fete eee ne 
sti is, of course, not surpris 
stir > 
» nulus trace was suggested by 
"rni A "fest f 

ning which manifest 


twee Lewise 
veen the CS and UnCs. likewise, 


f the reve circuit is a comparatively 
psychological \ 
Hartley (17° 
ved, the sens 
For an € 


‘ 
Alth 
ough the concept ° 
I has long been known. For 


new 
one à 
eom me, the corresponding 
bi aple, Boring (1929) cites 
ce = 
ject of sensation has been reme 


but 
become fainter the while" (p- 199) - 
th special reference to rev 


on and its vibrations persist briefly 
xtended discussion of the concept 


ati 


of in ; 5 

immediate . ahe E 3 

oct nediate memory. W! erberating circuits, see Overton 
58). E 
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different—and not wholly apparent—reasons (sec Section IV), has 


stressed this phenomenon. In one place in his 1943 book he says: 


The first | principle] to be observed is that afte as | 
to act upon the receptor, the afferent impulse (s) continues its activitv for 
some seconds, or possibly minutes under certain circums 
gradually decreasing intensity. 
cally important because it bring: 
environmental events which arc 
have occurred in the 
Thus is effected 
(p. 385). 


r the stimulus (S) has ceased 


tances, though w ith 
This perseverative stimulus trace is biologi- 
s the effector organ ev rapport not only with 
occurring at the time but with events which 
recent past, a matter frequently critical for survival. 
a short-range temporal integration (Postulate i, pe 47) 


Farlier in the same chapter from which Postulate 1 comes, Hull 
Says: 

Certain molar behavioral observations re 
the after-effects of receptor stimul 


ous system for a period measur: 


nder it extremely probable that 
ation continue to reverberate in the nerv- 
) able in seconds, and even minutes, after the 
termination of the ction of the stimulus upon the receptor. Apparently the 


receptor after-discharge -< is much too brief to account for the observed 
miata daga Lorente de Nó (1933) has demonstrated histologically the exist- 
ence of nerve-cell org: night conceivably serve as a locus 


: anizations which 1 
for a continuous circular self-c itation process in the nerve tissue. Rosen- 
blueth (1934) has presented experimenta] evidence indicating that neural 
after-discharges affecting the heart rate May persist for several Tifüntcs This 
is shown graphically in | Fig. 10-10.] Such bits of evidence um these tend to 
substantiate the stimulus trace by pothesis of Pavloy (1927 n T a 0) which 
is utilized extensively in the present work. Indeed, ore jc t fot the pre- 
sumptive presence of stimulus traces it would be impossible to account for 
ar behavior, notably those involv- 


whole sections of well-authenticated mol 
ing the adaptive timing of action (pp. 


41-42). 
7 Fig. 10-10. “Graphic represen- 
T T tation of the reflex slowing 
6 down of the heart rate of a 
2 cat and its recovery after ces- 
s E 4 sation of the stimulus. Left va- 
$4 Eus and depressor stimulated 
= centrally; right vagus cut; ac- 
oo 3 cellerators intact. Ordinate: 
= slowing of the heart per 10 
E ? seconds. Stimulation was at 
1 the rate of 78 Per second dur- 
> ing the period indicated by 
0 | | L t= em the h " portion of the base 
9 50 100 150 200 250 300 350 400 450 line. (Adapted from Rosen. 

Time in seconds 


blueth, 1934, Fig. 6A, p, 


303.)” 
(Hull, 1943, p. 43.) 
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Vhat Hull realized that the phenomenon of stimulus trace does not 
resolve the problem of delayed primary reinforcement af 
in his theoretical system is indicated by the 
he accepted the Spence-Grice inference that 
a gradient of primary 
or five seconds. 


however 


a "habit" as conceived 
readiness with which 

reinforcement or that 
Hull was well aware 
may last for a considerably longer 
as dependent upon conditioning 
connections, it would 
al reason whatever 


there i * 
there is no such thing as 
it |: X RE ; 
ae at most, for only four 
[0] + NG Á, * 

the fact thar stimulus traces 


time; and if he had seen learning 
of S—R "habit" 
heorctic 
nglv 


rather E 
ther than the strengthening 
at there is no t 


ft 
ave Nay 5 
€ been apparent to him th 
of correspondi 


fur dini pie A 
dismissing the possibility long gradients of 
The immediately 


hire Fant 
primary. reinforcement. 
“time” in 


litional observat 
icates that for 
not instantancous 
rediate memory). 
Y this rendency 


ion is in order. 
living organisms 
but, through the agency 
has some considerable 
objective stimulus 
possibility of con- 
and conditioning, 
q vision into the 


Mes passing, one ade 
oe discussion indic 
f Seca the present. 15 
m QNM dee uA 
M E > duration. Witl 
dichas persist briefly in the 

g is, as we have seen, cn 


for 

nervous system, the 

P ormously increased; 

h learning, thus gives us both memory of the past anc 
iture, E 

of the phenomenon of 

for gradients of 


f further evidence 


importance 
other thing 
o a review o 


the critical 


St. sl 
O again we scc 
among 


basis, 


ee 
Umulus trace, as the 
turn now ¢ 


Srt menie 2 
Primary reinforcement. We 
for sucl 1 
such gradients. 
VI 

: Delayed Reinforcement Type-D; 


Extrinsic Stimuli 
ave been ob- 


Let us examine, now. experimental results which h 
the interval, not between a response and an en- 
ironmental stimulus and 


nv 
by Roberts, already cited 
an early investigation of 


tained by varving 
Sung reward, but bet 
of affairs- 
reference 
icd out by 
ments reported in Animal Intelligence 
i so familiar that the briefest mention will suffice. One group of these, 
however, secre. hace failed to attract the attention that it deserves or to 
hive been given irs proper weight in the discussion of theories of learning. 
* eu Professor Thorndike would say 1n 3 firm tone, "I must feed those cats." 


en seconds later he would feed them. The w ords came in time to stimulate 


ween some external, c 


In the paper 
is made to 


4 rew s 
rewarding state 
Thorndike. Says Roberts: 


in thi i 
" this chapter, passing 
tis nature which was call 


ical experi 


"horndike's (1911) € 


As to hether or not this indicated the presence 


the c: : i 
be ate eS ted aetivity- F 
ideas,” the investigator apparently could not feel quite certain (p. 49). 
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ie is pc E Da. . 
Then, some years later, j avloy 
ported observations of à similar 
present a stimulus, such as 


and then, after an interv: 
Pavlov: 


(1927), using dogs as abject z 
kind. Here the procedure w Lak 
a buzzer or light, for one-half to one i et 
al of one to three minutes, present food. Says 


K P All not 
It is found that after Several repetitions of this routine the stimulus bod 
itself evoke any reaction; neither w ill its disappearance; but me Apps i s of 
reaction (salivation ) will occur after a definite intery al, the pam s ti 
the excitation caused by the sti ing the operative factor. We bs hich 
distinguish this y > type described prey iously, 2j cU 
the unconditione or part of its duration with the id 
ditioned stimulus, The type described in this paragraph is termed a 
conditioned reflex (pp. 39-40). 


Pavlov then reports an experiment b 
Was fed regularly at 30-minute interv; 
that, despite omissi 


tion of saliva With a Corresponding 
duced at about the thirtieth minute, 
late” (p. 415; cf. Liddell, 
Interesting and suggestive 
the data obtained Were not very e 
purposes of determining the full e 
gradient of reinforce 
turn to other 
reported to date is th 
Jenkins. After briefly: 
in this chapter, i 


N à : ich a dog 
y Feokritova in which a € 5 


È eer: : j found 
s. After training, it was f vh 
established interval, "a sec : 
3 R ic DEUS 
alimentary: motor reaction is p 


i o minutes 
but it may be one or two minu 
1950, P- 194). 


as the experiments just described at 


xtensive or systematic; and for 
tent and shape of the complete 
learning, it is necessary to 
gly best-controlled studies 


: f 
and published, in 1950, by W. O. 
to some of the 


le seemin 
at carried out 
alluding 
Jenkins says: 


studies already discussed 


Separate from these 
the operation of tempo 
That is, how clos 
in order for it to 
The present expe 


Problems but relate 


ral factors in the 
€ to prim 


acquire the 
riment de; 
rmerly-ne 


forcing properties. 
st a stimulus occur 
the response by itself. 
reinforcing properties 
1 of the time-interval 
cinforcing agent. T he 
d in terms of learning 


v of maintaining 
acquisition of 
> as a functio, 


1 Stablished r 
cing agent Was measure 
a new response (p. 238). 


and performing 


Preliminary training 
individually, into a small comp 
square, sounding a buzzer for 


after an interval, by food. 
into five groups ( ranging 


d in putting them, 
artment something less than a foot 
three seconds, and then following it, 
Subjects, before training, were divided 
in number from 10 tO 15 rats per group), 


The 
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for these five groups was, re- 


and the " 
he buzzer-food interval, or delav. 
The rats were maintained at 


spective 
pectively, n 


Beis oe 
3 * of nor : 
mal body weight throughout th 


procedure i E : 
ure is described as follows: 


Scd. $us and 81: scconds. 


c investigation. The testing 


tion of conditioning. the food magazines 
food following occurrence of 
] in the boxes. Depression of 
s. A toral of 6 hr. of 


al time of ex- 


i r 
uet kd HOY follow ing comple : 
buzzer | S i the appo pre interval, to deliver 
the levers - c Abonnee and the levers insertes 
nr-pressin us uced the buzz and activated the recorder 
T | Agen ith a half-hour session per day at the usu diini 
g. Food was never presented in the experimental boxes after the 


ever had b ; 
had been inserted (p. 239)+ 


Fig. 10- TET 
dry n. Gradient of second- 80 
V reinforce s e a 4 
lies orcement, 1l ype-D. E 60 i 
fis ME shows bar pressing = 40 
ia ats) which was reinforced [3 20 
Y à buzzer d s £ 200 
Vi that had been pre- $ 
ously foll ù z 2 80 
OREA ON ed by food at the $ 60 
Rer intervals indicated. 2 40 
oduce e ; 
l9so duced from — Jenkins. = 20 Ll E 8 
39. p. 240.) 100551 10 20 30 40 so 60 70 80 90 
Time interval in seconds 


The function thus 
asis of superfi ally 
1 Perkins. In dis- 


lly in Fig. 107! [^ 
1, on the b 
bv Perin anc 


1 graphica 
to that obtained 
rocedures, 


The à 
he results are show! 


obtained ; : 4 
va ined is quite similar 
Vv differ s 
Tr ditferent experimental p 
ss r3: a : 
Ing his findings. Jenkins says: 
a pronounced and significant temporal 
] by pairing 3 neutral stimulus 
ts for the derived reinforce- 
to distant remporal associ- 


SIS study has demonstrated pow 
With "E of derived reinforcement Dg jedem 
me PER reward in conditioning. Respons 

nt differed by nearly a factor of TWO from close 


ati $ : r 
Bi of the two stimuli [sec also Bersh, 195! |. WS ‘ 
uring the training I appears that there were two stimuli to W hich re- 


5ponses had to be learned: the noise of the food-machine and the buzz. Àn 


attempt was made to bring the rats of all groups UP to an equal level as regards 
i r to buzz-food training by giving 20 pre- 


in ese the magazine prio p : ; 
lary trials. The latency data at the end of training suggest approximate 
ro the magazine. [It was necessary, of course, to have 

ild announce the appearance of food in, and which 
uds bring the rat tO: the food trough; for otherwise, the interval between 
duzzer and delivery of food might be quite variable, since the rat would not 
a "know" exactly when food became available. The sound of the food- 
delivery mechanism served this purpose. | The learning of the response to 


Cquality d 
ea in response 
de signal which woe 


370 LEARNING THEORY AND BEHAVIOR 


buzz presumably differed 
interval groups, it came 


ý pres 
R TEN r the she 
according to the time-interval, For 


: ine. 
F : n -Dagazu 
to serve as a cuc for approaching the i one 
: $ = shed the mag: 
Although the rats in the longer-interval groups also approaches 
after buzz 


; avec 
jin = wma would ! 
it seems reasonable to assume that further training EST cade 
$ i iati 1 E erar »robabh 
produced a greater differentiation of behavior. A factor that l 


: and 
as ane fe he buzz : 
the response to buzz more difficult to learn was thz similar ity of MR chic latter 
the presentation of the food magazine, Both were noises Ri Vile ager 
exis OP ia š "s acted by s 
lasted less than 9.5 sec. The noise of the Magazine presumably ac 


: "red even 
Puer gs a curred e 

of generalization to maintain approach responses when buzz occ 

or the long-interyal groups (p. 242). 


If the Jenkins ex 
Was this: the stimulus Which 
(food) at Various inte 
announced the 
mediately: associ 
ably some gene 
sound to the 
level of respe 
del 


ss, TE 
periment may be said to have had a Se a 
Was paired with primary reinforce hich 
rvals was somewhat similar to the stimulus wh 
arrival of food and Was therefo 
ated therewith, As Je 
ralization of sc 


former one; and tl 
nding, 


re, in all groups. i 
nkins points out, there was pro F 
ondary reinforcement from the d 
\ account for the relatively high 
as sole re 
ay groups. It Would have been prefer: 
such as a blinking light, in place of the 


down on this type of Seneralization, 
pears to have been admirably 


still vields a Curve which is ver 
of experiments Previously 
shortly, to be the “true” 


uis may y 
with buzzer ‘ inforcement, in the longe 
able to usc a visual stimulus, 
buzzer, as a means of cutting 
Otherwise 
designed; and c 
"V similar 
described 
gradient of 


the experiment ABC 

Yen with this defect, » 

to what Appears, on thc d 

and also those to be discussec 

delayed primary reinforcement. 

VII. Delayed Reinforcement Type-l, 
Intrinsic Stimuli 


At the outset of t 
experiments that 
decremental reinfo 
mental response; 


his chapter, 
have been car 
TCement follo 
and in the 


We conside 
Tied out 
Ning the 


red some of the aA 
9n the effect. of delayec 
perform 


ance of some instru- 
' immediately preceding section, we have 
seen that a very similar gradient, or function, is obtained when thc 
delay is varied between some extrinsic stimulus an 
turn now to 


a consideration of 


d a reward, Let us 
the interval between a Particular 


results obtained in situations where 


: response and a punishment has been 
systematically” varied. Here the index of learning will, of course, be 
response inhibition rather than response strengthening or facilitation. 

In an experiment ų hich Mow Š 7 


ert 
a somewhat diff, 
at the particu] 


eral vears ago, for 


& Ullman (1945) c 
was found th 


crer 
ar bit of be 


arried out sev- 
Mt purpose (sec Section IX), it 
havior on the part of rats (i.e. 
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1 not be much influenced 
was delayed by as much 


motivation (hunger) was 
provide 


ation coulc 


cating ; 
by Ses which was under investi 
^y. COMI x ii 
as nine paratively weak shock if the latter 
adim 
cconds, However, the competing 


fairly str . 
a toon rdg this situation, so that thc experiment did not 
by this ae A indicator of the extent to M hich the stimuli produced 
the leur y a behavior became conditioned to fear as a result of 
ence of the ensuing shock. 
n - relevant experiment 
Viek (Davitz, Mason, 
ain female rats, 
below 


Maite n 

has to vue another and more specifically 
Mowrer carried out in collaboration with Peter 

re & Vick, 1957). In this study 28 Lashley-stt 
Normal body A 3 reduced to and held at 15% € 
and unrest i bi eight Likes the weight of the animals when ona full 
seven ory ricted dict). Vhe animals were then randomly divided into 
Proper, ups, of four animals cach. On the first day of the experiment 

. cach animal was put into a box-like compartment approxi- 
‘The sides and top of cach compartment were 
a metal grill which could be 
the interior of cach of 
and there was à 


abor x 
it ; 
four months of age were 


mate] N 
ars gi 2 inches square. ) 
duele i and the floor consisted of 
these qus A small electric lamp illuminated 
apartments (of W hich there were seven). 
compartment. 
ent of this kind for three 
w) was offered 


Small 
all observari h : Y r 
servation window 1n the top of each 


Tos Pod animal had been in 
to it us dpud mash (made of Purina Laborat e 
anima] f Ta end of a stick, W hich was projecte. ae 
immedi rom beneath the grill. On the first day. Cac à 9 m jt 

iately ate the food, and the was promptly with- 


di tick 

ien. TER. i i ape Ai EA i 5 
ea Fhe animals, upon cating were then given a two- 
Au volts a.C 


a compartm 
tory Cho 
ard toward the 


e 28 animals 


conveying S 
the food, 
with 


10, 30% 


a limiting resistance of 
120, OF 600 seconds, 
animal has been as- 


“trial.” and 


bna peas shock (130 [ 
hms), after a delay of 9. ?* S 

; which a par 
and shock 
day. After a tri 
: hours; then they 
o hold their 
h of the seven delay intervals, 
The experiment as a 


ticular 
constituted a 
al all animals were left 
were returned to 


densos: 

o oe on the group to 
Onhy* ak ne presentation of 
Mà € trial was given per 
three 
ficiently € 
cacl 


food 


In tl 
we; apes 
€ apparatus again for ! 
weight at the de- 


their 
rd D Gan and fed sufficien! 
Were exel Seven animals, onc for Cac ^ ie 
di concurrently. for a total of pp de THE CS 
therefore consisted of four replications of this procedure, 
he experimental compartments 


animals int 
ach trial was this. Vhe object 


ad after € 
what extent t 
of the proximity or re- 
as most important that 


They R í 
Ye reason for leaving the 


for 
r such ; : x 
ch a long period before at 


» sec TS. he subjects were de- 


Of the ; 
le experiment was t 


terred f : 
ed from taking the food as 
"Therefore. it W 


a function 


Mote 
ne É * 
ess of the punishment 
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i sperimenr: . art- 
the subjects be thoroughly habituated to the experimental compar | 
ments, Le. not afraid of thc compartments as such. If the rats hac 


; : . 3 Mc and 
been put into the compartments and immediately offered food an 


then shocked at the various intervals, it is likely that they would be- 


compartments and thus inhibited 
from eating, regardless of the interval by which punishment of the 
specific act of eating was delayed, However, having them in the com- 
partments for a total of six hours each day ensured extinction of the 
fear reaction to the compartment itself and guaranteed that fear of 


eating would be specifically related to the nearness or remoteness of 
the shock. 


come afraid as soon as placed in the 


There are obviously two ways of summarizing the data obtained 
from this experiment. It can be done in terms of 
hibitions," i.e., the number 
ment that cach animal refrained from cating (for 
onds, after which the food w 
one can summarize the data 
cach animal on all trials. Both 
the results are shown graphical 
gradients thus obtained cor 
those of Perin, Perkins 

There was, however. 


the “number of in- 
of times during the course of the experi- 
as much as 10 sec- 
as withdrawn and the “trial” ended), Or 
in terms of the total. delay 
of these methods have been 
ly in Fig. 10-12. 


shown by 
used, and 
As will be seen, the 
nparc favorably, in shape and extent, with 
» and Jenkins, already. discussed, 


* à possible source of weakness in this experi- 
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Fig. 10-12. Two measures of inhibition for se 


tervals between eating and shock (Davitz, Ma 


ven groups of rats with Various jn- 
son, Mowrer, and Vick, 1957, p. 72). 
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Ment whi 
x hich TE: 2 : P € 
prevented publication of the results for more than à 


decade , 

cade. It will be recalled that the interval which is here systematically 
taking and eating food and the 
in the form of electric shock. 
t to be valid, 


Cxplored 3 
oe = that between the act of 
Ice of a subsequent “punishment” 
reinforcemen 
in any unrecog- 
cinforcement 
an artifact 


Wo] < 
it is ee yor any study of delayed ) n 
nized or pate the delay interval not be “bridged, 1 
Postponment abe way, s0 as to make the effect of r 
(on the basi coe marked than it would be without such 
source of hee ier stimulus trace alone). As we have seen, 
culty in the Jenkins experiment (Section \ Diti 
on of secondary reinforcement from 
) which wa immediately 


a possible 
s that there 


may: | 
ave " j B 
Y have been some gencralizati 


the 1 

sti 3 
folloy mulus (sound of delivery 

ed by food "back" to the stimulus (buzzer) which preceded 
the ditferent experimental groups). 


did not arise in the study just rc- 
possibility that a small quan- 
the subjects on each daily 
riod. If this hap- 
tc thereof) would be directly 
) vary the interval 
vould be vitiated. 


mechanism 


foc 3 
Xe by vari 1 £ : 
TERR V various time intervals (1n 

particular type of difficulty 
one: the 


porte 
d. but there was another 
] to 


tity * 
"ial" i. food which was presentec 
pened, iin Hu their mouths during 
associ ood-in-mouth (or mere tas ET 
all groups. and the effort tc 
d the occurrence of shock v 
ain in the mouth after 
ic would be 


ated with shock in 
cating would 
come progressively 
what like the one 
“stimilus trace" 
chemical 


jo 
Eo ‘aking food and 
be pineal which might remain 
ssolved gradually, the taste residu 
: provide a gradient some 
casoned that the 


it could be r 
was in this experiment 


CSS ; 
odis might, therefore, 
from 2. In other words. 
Düler ye food and act 

WA than. neurological. 

i ut even now, no guarantee that the lir i 
es id id not operate as just SU Hui an el i 
ins onducted for a different Pt ee Section IX) and in which 
ame source of difficulty cou existed, Bixenstine ( 1956) 
(though tic) results: an electric 
s after cating had a decidedly inhibitory 
ereafter had a very much 
ct (sec also Bevan & Dukes, 1955). 
the. Vick study give gradients suf- 
arches previously cited and in 


of eating 
agering taste of the 
iggested. seriment which 
urpose (S! 
: Id not have 
bei very similar less systema 
uc eU 20 seconds 
guis ce, and a shock coming § 
Mires but still noticeable effe 
aai the results obtained in the 
ise d like those obtained both in pes i ! 
€ to be described in the next Section, so that the question of 3 
Possible artifact here is nof in any case 
1959). : 


50 seconds th 


a Sey crucial one (cf. Kamin 
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VII. Delayed Reinforcement Type-l, 
Extrinsic Stimuli 


Thus far we have reviewed three types of delayed reinforcement 
experiments. We have discussed experiments in which there was à 
variable interval between cither a response or an environmental stimu- 
lus and a reward; and we have just considered an experiment in which 
there was a variable interval between a response and an ensuing pun- 
ishment. The remaining question is: What is the effect of varying the 
interval between an environmental stimulus and "punishment," i.c. 
primary incremental reinforcement? Two rather ditferent experiments 
will be described, one in some detail, the other more briefly. 

The first of these studies, on the effect of varying the interval be- 


tween an external stimulus and incremental reinforcement ("punish- 


ment"), was reported in 1954 by L. J. Kamin. It is entitled, “Traumatic 
Avoidance Learning: The Effects of CS-US Interval with ‘Trace-Con- 
ditioning Procedure.” After citing an early attack on this problem 
by Warner (1932a), Kamin says that the objective of his own study 
Was to explore “the effects of CS-US interval on the 
shuttle-box avoidance response | see Chapter 2 
There were four reinforcement-delay interv 
seconds, with the interval measured from the 
buzzer to the onset of shock which stayed on until the animal 
“shuttled,” i.e., jumped over a low hurdle into the other half 
apparatus (sce Fig. 
from the em 


acquisition of a 
in dogs" (p. 65). 


als: 5, 10, 20, and 40 
onset of a two-second 


of the 
asured 


.c., the truc 
ach instance two seconds less than the delay 
values given. The electric shock used with each dog was just below an 
intensity which would be tet 

avoidance learning, 


12-19). Thus, the four intervals used, if me 


7 of the buzzer to the beginning of the shock, i 
“trace” periods, are in ¢ 


anizing—hence the expression “traumatic” 
The interval between trials 


ranged between two 
and four minutes, with a me 


an of three minutes. 
Training was continued, within a single session, until each animal 
had made five successive “avoidance” jumps. The chief 
cedure, involving resistance to extinction, 


Three minutes after the dog's fifth consecutiv e avoidance 
were begun. The CS was still presented 
the US was never again applied. 


testing. pro- 
is described thus: 


extinction trials 
al intervals, but 


at the same intertr 


| r E Ven extinction trials w ere conducted on the 
first experimental day. They were followed by a 5 min. "cooling-off" period 


for the dog. The dog was then returned to its home 
all succeeding, experimental days follow 
tinction trials. The 


age. The second, and 
of the first day’s ex- 
as followed by ten. CS 


ed the pattern 
10 min. observation period w 
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presentations and by a 5 min. cooling-off period. Thus, each daily extinction 
The extinction trials were continued until the dog 


session lasted 42 minutes. 
cutive failures to respond to the CS 


had achieved an extinction of five conse 
by jumping. However, if extinction had not occurred within ten days (100 
trials), the animal was discontinued (p. 66). 


Fen mongrel dogs served as subjects in cach of the four experi- 
two control groups, of five animals 
to sce to what extent the behavior 


of the experimental animals might have been a function of the presenta- 
tion of shock alone, without the occurrence of the warning signal. 

as found "that. the number of trials re- 
ng| increases with the CS-US 


Mental groups; and there were 
cach, which were used in order 


As might be expected, it W 
quired to reach criterion | of. conditionit 
interval" (p. 67). However, the data which are most clear-cut and in- 
teresting for present. purposes are those derived during extinction. 
inction curves for the four experimental 
total extinction performance of 
S-US interval. The similarity 
previously reproduced in this chapter is 
still further one’s confidence that, re- 


Figure 10-13 shows the ext 
groups; and Fig. to-t4 shows the 
these four groups as a function of the C 
of the latter curve to those 
striking and serves to increase 
gardless of whether the antecede 
mental stimulus and regardless of wl 
is deeremental or incremental, the gr 
or upwards of 


nt event is a response or an environ- 
vether the ensuing reinforcement 
adient of reinforcement is nega- 
tively accelerated and lasts f 30 to 6o seconds as a 
limit, 


100 T 


80 


5 sec 


Fig. 10-13. The course of ex 
tinction of an avoidance Te- 


sponse in four groups of dogs 


60 


10 sec 


Wherein the danger signal had 
xired with 


40 


Percent CR's 


Previously been 
Shock at the intervals indicated 


(Kamin, 195 
s + 1954, pP- 67)- 5 
20 


Extinction days 
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The only other investigation known to the writer in which a pe 
tematic attempt was made to study the effect of varying bg a 
between a danger signal and the painful event thereby SHUT ik E 
carried out by Davitz & Mason (1957).^ The details of this Acme 
will not be given here except to say that, using rats as subjects, osi 
investigators paired a two-second blinking light with a Hore 
electric shock with various interstimulus intervals (the same as those 


100 1 

80 Es 
E | 
= 60 - Fig. 10-14. Resistance to extinc- 
E tion (“percent CR") as a func- 
H tion of training interval between 
Š 49 CS and shock (Kamin, 1954. p. 
= 67). Cf. Fig. 10-13. 
g 
E 

20 


9 $ i0 20 40 

CS-US interval in seconds 
employed by Mowrer & Vick). The extent to which fear was con- 
ditioned to the blinking light was then 
this stimulus to inhibit exploratory 
findings are in essential agreeme 


measured by the capacity of 
locomotion in a test situation. The 
nt with those of Kamin, just described, 
and with those reported by Perin, Perkins, Jenkins, and by Mowrer 
& Vick, using different procedures. 

In conclusion Davitz & Mason say: 


On the basis of the evide 


acquisition of instrumental responses, Spe 
der conditions of delayed rew ard is 
ondary reinforcement. The e 
study, however, precluded the possibility of se 
operating during the delay period. The gra 
more parsimoniously accounted for in 
and casts some doubt upon the 


nee from a number of experiments concerning the 


nce proposed that all learning un- 


a function of the effects of immediate sec- 
Xperimental procedure utilized 


in the present 
condary reinforcing cues 
adient obtained does not seem to be 
in terms. of secondary reinforcement, 
adequacy: of Spenee's hy pothesis (p: 73): 
"For a related approach 


«with similar results, see Brown 
(iggob). 


(1939) and Mowrer 
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istently to the con- 


Fhe evidence thus seems to point quite con 
clusion that both decremental and incremental reinforcement, follow- 
al stimulus or a response, can be delayed 
30 seconds and still result in some degree 


rogressively greater as 


ing either an environment 
in mammals for as long as 
of conditioning. with the effect becoming p 
the interval between stimulus, or response, and reinforcement is 


shortened (see also Dmitriev & Kochigina, 1959)- 


IX. The Problem of Temporal Integration 


oral gradients of reinforcement (made 


Phe biological utility of temp 
) is obvious: since a re- 


possible by the phenomenon of stimulus trace 
sponse is more likely to have produced (ora stimulus to be a sign of) 
g event which occurs immediately therc- 


some rewarding or punishin 
event of a 


after than it is to have produced some temporally remote 
similar kind, it is clearly an advantage for temporally proximal events 
to have more of an effect. with respect to learning, than remote ones 
(sec also. Mowrer 1960, Chapter 8). It is nevertheless true. that. the 
effect of an action performed at onc point in time may be considerably 
delayed; and when this happens unfortunate complications may ensue. 
It has already been noted, in connection with the discussion of Roberts’ 
experiment on delayed rev able mislearning may re- 
sult under these circumstances: irrelevant actions will 
ene between th 
other actions, 
reinforced. 
in that when, as 
“encouraged,” the very 
ans that they will also 


rd, that consider 
ie, since other, 
c truly effective response and 
arer to the reward 


almost certainly interv 3 
" coming nc 
But, as indicated, there is 


a result of such 


the resulting reward, these 
In time, will be more strongly 
à self-correcting provision he 
spurious reinforcement, irrelevant 
fact of their irrelevancy and ineffectiveness me 
tend to be extinguished. 

P another 


laved reinforcement a 
: so reinforcing consequences. 


occurring at different points 


in time. Consider this example. Suppose a buzzer sounds, indicating 
that food is now available to 2 hungry rat in a little trough; but sup- 
pose, also, that there is a "rule? which says that the rat is not to Cat the 
food for three seconds after its appearance and that the punishment 
for so doing is an clectric shock which comes. let us sav, 10 seconds 
after the “taboo” has bec As an experiment reported by 


Mowrer & Ullman (1945) indicates, it is virtually impossible (under 


actions are 


ation where the complica- 


type of situ 
re not so benign. This is 


| There is, however. 
tions resulting from de 
Where a given action oF 

one a reward and the othe 


response has tu 
rA punishment 


n violated. 
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the conditions described) for rats to find the integrative aluran Ee : 
problem of this kind, which would be to wait briefly ane EST 
has appeared (three seconds or more) and then cat the food. à 
these subjects do instead is to cat the food, immed 


ately upon its ap- 
pearance and then get punished later, without ever “seeing the con- 
nection.” AIDS T 

Or it might be that the situation is reversed: there could. easily 
be a situation in which there was a slight immediate punishment which 
would prevent the subject from getting to a larger. remote reward, 
just as in the experiment cited a slight immediate gain (immediate vs. 
slightly delayed eating) wins out over a larger remote punishment 
(cf. Muenzinger's research on "courage," described in the following 
chapter). 


Thus, through the uneven distribution of reward and punishments 
in time, the basis is laid for what Shaffer (1936) has called "persistent 
nonadjustive" behavior. And Hearnshaw (1956), in a presidential ad- 
dress to the British Psy chological Society, entitled 


“Temporal inte- 
gration and behaviour,” has said: 


Animals, even the 


primates, live mainly in the present. They are sense- 
bound: learned re 


actions are tied to present stimuli, or subj 
delay; anticipations are brief. Kohler’s ( 1925) observ 
many years spent with chimpanzees leads me 
besides in the lack of speech it is in the 


ect to but short 
tion is classic: “A great 
to venture the opinion that 
extremely narrow limits (in time) that 
the chief difference is to be found betw cen anthropoids and even the most 
primitive human beings." In confirmation of this observation we 
temporal maze is a peculi rly difficult task for 
for delayed response is f. 


find the 
capacity 
17-19). 


animals; and that the 
ar less than with quite voung children (pp. 


How, then, do human beings achieve their unique capacity for 
“time-binding,” as Korzybski (1933) has called it? Rather clearly the 
phenomenon of language (Mowrer, 1960, Chapters 3 and 4) is cru- 
cially important here, But certain animal studies give at least intima- 
tions of the direction in which the solution to this problem lies. 

At the outset of this chapter we reviewed a number of experi- 
mental situations in which the temporal gap between an action and 
a delayed primary reward was subtly spanned by an inadvertant ere 
(secondary reinforcer) of some sort. And in Chapter 5 passing refer- 
ence was made to a study in which Wolfe (1936) taught chimpanzees 
to "work" for poker-chip "tokens" which could later be exchanged 
for food. The latter was thus a deliberate attempt. to : 
effect which had occurred accidentally: 
(one of which, incidentally, ' 


produce the 
in several of the early studies 


had been performed by Wolfe himself, 
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nforcement. The trick, in Wolfe's 


1934) involving delayed primary rei 
ils to subject themselves to a 


poker-chip study, was to get his anim: 
(light “work” of some sort) as a 


slight immediate inconvenience 
rather. considerable but belated 


means of obtaining à promise of 
primary reward (food), Without the 
never have sensed the relationship; but with the aid of the tokens, 
/ reinforcement, the animals read- 


“tokens” the animals would 


Which provided immediate secondary 
solution to the problem. 

along similar lines has been reported 
was to show how rats, in the 
the "temporal integra- 


ilv found the "integrative" 
More recently, an experiment 
by Bixenstine. (1956). His objective 


Mowrer-Ullman type of dilemma, could solve 
ded by a cuc, or "token," of some sort. In brief, 


did was this: W henever food was presented 
rule of waiting. à little light “blinked” 
n just before the delayed punishment 
where Wolfe had provided his 
or promise, of reward to come 
his subjects an immediate 
later for present be- 


tion” problem if 
What this investigator 
and the subject violated the 

briefly and then blinked agai 
(shock) occurred. In other words, 
subjects with an immediate token, 
later for present behavior, Bixenstine gave 
"of a punishment to come 


token, or “promise, 
learning" resulted: 


havior. Again, “integrative 
When Ss are aided by à sign having clear, unambiguous reference to 
temporally remote, punishing consequence their integrative capacity Is 
multiplied immeasurably. We may say that the econdary drive properties of 
the sign (blinking light) at the time of its occurrence during the taboo are 
the crucial determinants; the point in time where this learned drive is rem- 
forced (ie, the remoteness of the blinking light and shock combination), 
though noteworthy, is only of secondary importance (Bixenstine, 1956, 
pe 165 ).5 A 


lacking in the ability to supply 
themselves with the tokens, oF “symbols,” which make integrative 
problem solving possible, certainly they are severely limited in this 
connection (scc Ferster, 19535 Mower, 1960. Chapter 2), whereas man 

But while this accomplishment 


par excellence. 
f his glory, it can apparently also lead to 


Although animals are not wholly 


is the symbol user 
is the touchstone of much o 
Serious. difficulty. 

not impossibly throw light on 
development and breakdown. 
d's personality develops 
. In breakdowns of per- 


[The study of] remporal integration may 
some of the primary problems of personality | 
Bowlby (1951) recently noted thar the young chil 
hand in hand with a sense of continuity in time. = 
a temporal (and spatial) interval, sec 


of a cue to span 
and Dyal (1957): cf. Duryea (1955)- 


For vet another use 
Mason. (1956). 3 


Grice & Goldman (1955)* 
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s l ^, both mild and severe, temporal disorganization is a common tranne 
s Ge consider les clochoards, the Parsian down-and-outs recently de 
ee s elder (1955), whose horizon is the idein S Bara do 
history, nor aspirations, but have surrendered unconditiona 5 w t o E 
of events. . . . If we are to explain behavior in irs most Earann a 
human forms without having recourse to animism it is to the caneepe 
temporal integration that we must turn (Hearnshaw, 1956, pp. 18-19). 


We cannot, of course, at this point pursue these inviting leads with- 
out getting very much ahead of our story; but it is useful here to 
have noted how quickly the study of so simple a matter as gradients 
of reinforcement, as observed in animals, carriers us into 
of human thought, planning, conduct, and adjustment (see also Chap- 
ter 8, p. 273). Elsewhere (Mowrer, 1960) we shall return to these 
matters, but in the meantime it will be our purpose to adhere to a 
More progressive, systematic, orderly development of a foundation 
upon which a full human psychology can firmly rest. 


problems 


X. Temporal Integration and 
“Superstition” 


In the experiments cited in the preceding section (Wolfe's use of 
poker chips as tokens of ultimate reward and Bixenstine’s use of a 
blinking light as a token of ultimate punishing), the “mediating” stimu- 
lus (cf. Mowrer, 1960, Chapter 2) was externally provided for the 
subject. Although human beings, in language, have a rich reservoir of 
anifestly limited in this connection 
as we shall presently sec) they 


such mediators, animals are m 

though ( 

resources 
As carly as 1948, Skinner published some highly signific 

tions in a paper entitled “ ‘Superstition’ in the Pigeon.” Because it was 


so compactly written to begin with, this paper is hard to abstract and 
will instead be quoted at some length. 


, al- 
are not entirely without such 


ant obser 


A pigeon is brought to a stable state of hun 
of its weight when well fed. It is put into an experimental cage for a few 
minutes each day. A food hopper attached to the ¢ e 
place so that the pigeon can cat from it. A solenoid 
the hopper in place for five sce, 

If a clock is now arrangi 
with no reference wh 
usually takes place. 
clearly defined th 


ger by reducing it to 75 per cent 


age may be swung into 
and a timing relay hold 
at each. reinforcement. i : 
ed to present the food | 
atsoever to the bird's beh, 
In six out of eight cases 
at two observers. could 


Yopper at regular intervals 
avior, operant conditioning 
the resulting responses were so 

agree perfectly in counting in- 
“For a discussion of similar 


problems in the cont 
conflicts," see Brown ( 1957); 


ext of what he calls 
also Rigby 


“temporal 
(1954). 
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ancously learned | to turn counter- 
turns between reinforce- 
upper corners of 


stances. One bird was conditioned [spont 
clockwise about the cage, making two or three 
ments. Another repeatedly thrust its head into one of the 
the cage. A third developed a “tossing” response. - » - 
ahs: conditioning process is usually obvious. The bird happens to be 
executing some response as the hopper appears; as a result it tends to repeat 
this response. If the interval before the next presentation is not so great that 
extinction takes plac second “contingency” is probable. This strengthens 
the response still further and subsequent reinforcement becomes more prob- 
able. It is true that some responses go unrcinforced and some reinforcements 
appear when the response has not just been made, but the net result is the 
development of a considerable state of strength (pp- 168-169). 


at is going on here, it may be help- 
on “hypotheses in rats." In the 
Skinner situation, it would appear that, to speak loosely, the pigeon 


" : 3 
assumes” or “infers” that the appearance of the food must be “con- 
and so forms this or that “hy po- 


ationships between behavior 


In order the better to grasp wh 
ful to recall Krechevsky’s 1932 paper 
th: 


Ungent™ upon something v hich it does 
adventitious rel 


thesis, depending upon 
count continues: 


and reinforcement. Skinner's own ae 
said to demonstrate a sort of superstition. The 
ausal relation between its behavior and the 

ion is lacking. There are many 
behaviors have, of course, no rcal 
an allev, just as in 
gecon did nothing— 


bi Fhe experiment might be 
dird behaves as if there were a € 
presentation of food, although such a re 


analogies in human behavior. «+ - These be 
effect upon one’s luck or upon a ball half way down 


the present case the food would appear as often if the pt 
did something else (p. 171). 


s. We come now to 
namely his comments 


Or, more strictly speaking, 
general remark 
aper. 
aved reinforcement. 


So much, then, for Skinner's 
the more specifically relevant part of his p 
and observations concerning the question of del 
of reinforcement. In general, 


the rate 
al, the speedier and 


intervening interv 
ason is that the pigeon’s behavior be- 


irs to depend upon 
shorter the 
ng. One re 
asses after reinforcement. «+ + 
flectiveness of short intervals is that the 
intervening responses emitted 
ancels the effect of an oc- 


The effect appe: 
We should expect that the 
more marked the conditionit 
Comes more diverse as time p 

Another reason for the greater c s 
onger the interval, the greater the number of 
Without reinforcement. The ting extinction c 
Casional reinforcement. 

so Fifteen scconds is 
dicated above. One min. is muc 
up, however, the interval can be length 
two min., and a high rate of responding W 


Weakening (p. 169). 


resul 


a very effective interval at the drive level in- 
h less so. When a response has once been set 
hened. In one case it was extended to 
as maintained with no sign of 


As of 1948 Skinner regarded the behavior here described as "super- 
prevailing circumstances, 


stitious.” nonfunctional, which, under the 
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it indeed was. But more recently Ferster (1953) has nde is 
cumstances are slightly altered, this behavior can become exu aordina "d 
useful. What Ferster did, first of all, was to show that if pigeons ie 
allowed to learn an instrumental “pecking” response with ia 
prompt reward, this response extinguished in three out of four e 
when the reward was consistently delaved for 60 seconds and that 
extinction occurred in the remaining bird when the delay was extended 
to 120 seconds. Then reported Ferster, 


The same Ss that extinguished under 60o- and 120-86C. delays w ere re- 
conditioned on the original variable-interval schedule without dela 1 Short 
delays were then introduced and the value of the delays was gradually in- 

sed as S's behavior became stable under any particular delay. It was 


possible in three out of four of the cases to sustain normal rates of response 
under delays of 60 sec. (p.224). 


Observation of the pigeons indicated th 


at, by virtue of the procedure 
just des 


ibed, they had developed “superstitious” 
delay interval and that these had become 
60-second interval between the origin 
tual food reward. Here, Ferster re 
response will not be maintained 

but depends on the food-reinfor 
other words, once an interval response has been adventitiously. rein- 
forced and thus established as "superstitious" behavior, 
apparently links, in some wav, the origin 
food reinforcement. Ferster seems to 

that of “response chaining’; and he reg: 
Supported by the fact that a delibe 
pigeons specified (and measur. 
the same Way as 


responses during the 
instrumental in bridging the 
ally pecking action and the even- 
asoned, the “strength of the pecking 
directly by the present: 
ced mediating behavior” 


ion of food, 
Ops 2235). In 


its occurrence 
al pecking action with ultimate 
assume that the mechanism is 
ards this interpretation as being 
rate attempt on his part to teach 
able) responses “which could function in 
a ‘superstitious’ chain conditioned ; 

the delayed procedure” (p. 224). But it is also possible that the delay 
interval was being spanned by the device of second; 
or, equivalently, second-order conditioning, If. 
response was "superstitiously" learned as 
imity to the (causally un related ) 


à consequence of 


' reinforcement 
.in other words, a given 
a result of its temporal prox- 
l appearance of primary reward (food), 
this would mean (a) that the habit itself (if our general conception of 
habit formation is valid) consisted of the secondary: 
became conditioned to the back-lash stimul f 
question and (b) that this stimulation, if 
imity to the truly instrume 
inforce that response, ic 
reinforced mediating be 


reinforcement that 
ation from the response in 


produced in temporal prox- 
ntal response of pecking, can secondarily re- 
5 Can serve, in Ferster’s Words, as “the food- 
havior” 


but does so by means. of secondary 
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reinforcement (as previously discussed in this chapter) rather than 
through the device of stimulus-response chaining. 
geons would show an increased 


) It would be interesting to sce if pi 
facility in developing this type of mediating behavior if subjected to a 
described by Ferster. It might 


number of different situations of the kind 
be that, with sufficiently extended practice in varied contexts, they 
would become relatively "expert in the use of such mediating (“super- 
stitious") responses (cf. Harlow's concept of "learning sets," 1949). 

Recently Noble & -Alcock (1958) have discussed the difficulty of 
cts the diminishing efficiency of re- 
instrumentally effective 


demonstrating with human subje 
Ward as the interval between the “correct.” 


response and the reward is increased. They say: 


There is a common belief that reinforcement and feedback in human 
ig, must be undelaved and specific if optimal 
307). | But] whether reward or information 
S docs during the time 


learning, just as in rodent learnir 
proficiency: is to be achieved (p. 407 
Is withheld seems to be of less conseque 
interval between response and after-effect ( p» 408)- 


nce than what 


They then quote Lorge & Thorndike (1935) as follows: 
-ent human subjects from keeping a connec- 
r-effect occurs or from recalling it to mind 
repeating some equivalent of the after- 


be immediate, regardless of how long 


, It is almost impossible to pre 
tion . . . in mind until the afte 
when the after-effect occurs and then 
effect. The real relation would then 
an ostensible delay we used (pp- 193-194): 
ith their remarkable skill in symbolic 
-establish, the “connection” between 
and punishment), even though 
time. For example, if a 


In other words, human beings, W 
Manipulation, can easily keep, or re 
ard or action 
separated in 
is responsible for a given (de- 


immediately upon completion 
a delay now, before 1 gc 


two events (action and rew 
these events are considerably 
subject "knows" that a given r 
layed) etfect, he might say to himself, 
of the response: “Although there will be t 
the reward, I nevertheless know | have done the right thing, so all I 
have to do is wait.” Such a sentence could undoubtedly mediate con- 
siderable secondary reinforcement. Or is it conceivable (though perhaps 
less likely) that the subject, instead of anticipating the rew ard verbally 
upon completion of the action, will wait and verbally reinstate the 
Action just as or just after the reward occurs. t 

f symbols (mediators), the pigeons 


atile in the use of 
ion had to use a relatively crude, but 


espe )nsc 


Being far less vers 
previously. describec 
not ineffective, procedure. 


action repeatedly in the “goal region, 


] in this sect 
If they performed some purely arbitrary 
“ie. shortly before primary re- 
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ward occurred, then the response-correlated stimuli took on S" eee 
erties; and if these stimuli are then produced (by the performance d 
this act) shortly after the occurrence of the rclevant, truly wai 
mental (but temporally remote ) action, they served to transmit a 
higher-order conditioning) S* properties to the stimuli produced by y ic 
relevant action and thus give it habit strength, so-called. Otherwise, 
without such a mediating mechanism or activity, behavior from which 
primary reinforcement is temporally remote will have no chance at all 
of being lcarned. 

In earlier sections of this chapter, we have reviewed a number of 
experiments in which the primary gradient of reinforcement was in- 
advertently bridged by some secondary reinforcer in the experimental 
situation. In the Skinner-Ferster type of situation, it seems that the 
subject himself—by means of what Hull has termed a “pure stimulus 


act” (Mowrer, 1960, Chapter 2)—becomes capable of providing a sec- 


ondary reinforcer which apparently serves much the same function. 


XI. Conclusions and Disclaimers 


Our review of the relevant literature shows that one or more in- 
vestigations have been reported for cach of the four types of rein- 
forcement gradient which revised two-factor theory suggests. There 
are studies on the effect of varying the interval (a) between a re- 
sponse and decremental reinforcement (reward), (b) between an ex- 
ternal stimulus (signal) and decremental reinforcement, (c) between 
a response and incremental reinforcement (punishment), and (d) 
between an external stimulus and incremental reinforcement (cf. Hil- 
gard, 1956. p. 153). Although the experiments here reported ar 
beyond possible criticism and alternative 
Spence-Grice hypothesis), the 
gradients of primary 


c not 
interpretation (cf, the 
results converge in suggesting that all 
reinforcement, both decremental and incre- 
mental, cover an interval of something like three-quarters of a min- 
ute, at most; and further analysis suggests that this common finding 
derives from the fact that gradients of reinforcement are based upon 
the underlying phenomenon « 


of stimulus trace, 
As indicated in Chapter 7 


7, there are four 
forcement. None of these seems as vet to 


studied as far as the problem of reinforcement dela 
Presumably: all of them Operate ina m 
has been found to hold for 
studies would of necessity 


kinds of secondary rein- 
have been systematically 
V is concerned. 
anner roughly equivalent to what 
primary reward and punishment. Such 
involve the phenomenon of second-order 


REINFORCEMENT AND TEMPORAL INTEGRATION 385 


g gradients might, accordingly, be ex- 


conditioning; and the resultin 
than do gradients 


pected to start at a lower level and extend less far 
i.c. primary reinforcement, Here, it 
arca inviting. further research." 
reinforcement has repeatedly 
her way: namely, as 


involving first-order conditioning, 
appears, is a relatively little explored 
However, the phenomenon of secondary 
entered into our discussions in this chapter in anotl 
an "artifact? which, if not carefully controlled, may easily and ef- 
fectively “bridge” the interval between stimulation and primary re- 
ntal efforts to establish true rein- 


inforcement and vitiate experime 
ances where there 


forcement gradients. On the other hand, in circumst 
is an unavoidable delay of considerable duration between stimulation 
(or response) and primary reinforcement and W here learning is none 
the less desired, it follows that secondary 
usefully employed (as Wolfe and Bixenstine h 
tional device (Mowrer, 1960. Chapter 2). 

If it be granted, as present evidence seems to warrant, that the 
gradient of primary reinforcement is, maximally, 30 tO 45 seconds in 
duration and if it be further : sumed that this interval is determined 
by the duration of stimulus traces. then the question arises: Why, in 
have living organisms developed nervous sys- 
venitude, rather. than markedly 
S his question must bc 


reinforcement. can be very 
a media- 


ave shown) as 


the course of evolution, 


tems which mediate traces of this m 
answer to t 
made to deal with it in the 


pter g). And certain other 
nforcement have for inte- 
be considered at another 


shorter or longer ones? Since any 
largely speculative, no attempt will be 
present context (but sec Mowrer. 1960, Cha 
implications which the facts of delayed rel 
grative problem solving will also have to 
tine (Mowrer, 1960. Chapter 10). 

note that in discarc 
all learning invo 
à common embarr: 
al between 4 stimulus 


ling the concept of "habit" 
Ives conditioning, revised 
assment. It has Jong been 
and a reinforcing state 


We should, however, 
and positing instead that 
two-factor theory avoids 
recognized that the interv 
of affairs, if brief. is normally bridged by the phenomenon of stimulus 
trace. But how, it has been asked, can th 
of affairs be spanned? 
rberate for a short time W ithin the nervous 
has thought it feasible to posit response 
writers. (notably Thorndike) 


c interval between a response 
j Nene is conc! at a 
ind a reinforcing state It is conceded that a 
stimulus can and does reve 


system, but no one, apparently. 
traces. Some 


reverberation, response i 
is problem by im 


have scemed to resolve th plying W hat amounts to 


1957b) has studied the effect of 
The results are congruent 


itten, Kamin (195 


Since the above was WT 
reinforcement type-l. 


delaying deeremental secondary 
With expectation. 
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retroactive causation. Others writers (viz, Spence and giai Mags 
taken a different tack on the problem; but because their ind z 
predicated on a continued acceptance of adlina baits t e i 
encounters serious difficulties. Because revised two-factor camna 
theory assumes that responses, in the sense of overt, behavioral acts 
are never “learned” and that all learning is in the nature of stimulus 
substitution, the concept of stimulus trace nicely handles all instances 
of delayed reinforcement. l l 

In certain types of conditioning (see, for example, Hilgard & 
Marquis, 1940), it has been found that the gradient of reinforcement 
is extremely short, with conditioning becoming practically non- 
existent when the CS-UnCS interval is over two or three seconds (cf. 
McAllister, 1953, Fitzwater & Thrush, 1956). This type of finding 
might at first seem inconsistent with the evidence cited in this chapter 
for believing that the gradient of reinforcement typically lasts. for 
something between 3o seconds and a minute. ‘The trouble, it seems, 
lies in the way in which the conditioning process is conceptualized 
and experimentally controlled. If one thinks 


of conditioning as in- 
volving mainly, « 


or perhaps exclusively, emotional reactions, then there 
is no difficulty in demonstrating the longer type of gradient; but if 
one thinks of a “conditioned response” as involving some overt re- 
flex, such as an eye blink or the knee jerk, then one is either dealing 
with an altogether different type of learning mechanism or with a 
very inefficient way of demonstrating the more basic phenomenon of 
emotional conditioning. This is one among many reasons why so 
little reference is made in this volume to the conditioning of the so- 
called “short-lateney” responses: they are of comparatively little bio- 


logical importance and do not, apparently, show learning in its most 
typical and important. form.” 


“After class one day 
they wanted to know 


group of students challenge 
€ not the short-l 
portant and just as "typical" and 


previously. stressed, By way of € instructor stepped up to one of the 
students and made a quick, threatening gesture toward his eyes. The student, 
of course, blinked and was then asked: “Why did you blink your eyes? You 
knew | wasn't going to hit you." 1 "he sald that he did indeed 


"know significantly enough, made no 

body or head). 
“involuntarily. 
ason for regarding the s] 


a primitive and nontypic 
are strictly “mechanical” and often occur ( 


“stupidly.” [n a few 


d the above position, Why, 
atency conditioned re 
interesting 
answer th 


i flexes just as im- 
as the other forms of learning 


A little sheepishly, 
that |. was not going to hit him (and, si, 
effort to avoid a blow by moving his whole 
just. "couldn't help? blinking. It occurred 
out thar this is precisely the re 
reflexes as representing 


D, bur added that he 
It was then pointed 
Short-lateney conditioned 


al form of learned action. They 
as in the fore 
situations, as with rapidly 


going instance) usclessly. 
Moving objects 


approaching the 
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Nor has any attempt been made 
the gradient of reinforcement and 
a literature on this problem, but to date 


to discuss the relationship between 
amount of reinforcement; there is 
it has not proved very in- 
structive, since it has been found that this variable is not usually very 
1). Within narrow limits, at any rate, 
reward or a small punishment “works 
19 put this fact should not be 


consequential (but see Chapter 
it has been found that a small 
reinforcements; 
of delaying reinforcement. As the several 
there is an or- 


just as well as “larger” 
confused with the effect 
ited in this chapter consistently show, 
highly efficient learning at zero OF brief delay 
at all (without "bridging" stimuli) when 
longer. This gradient, as we have seen, 
ii and mediated by the phenome- 
the action-reinforce- 


experiments c 
derly progression from 
intervals to no learning 
the interval is a minute € 


seems to be specifically dependent upon 
In some instances, where 


non of stimulus trace. 
e stimulus trace, and where this 


ment interval exceeds the duration of tl 
interval is not bridged by external stim 
(even though infr 


uli having secondary reinforc- 
ahuman) will develop “media- 
1960, Chapter 2) which serve the same pur- 
these responses occur but do not serve this 
lerstood by the observer), 


ing power, the subject 
tion" responses ( Mowrcr, 
poses. In situations wherc 
purpose (or where they are perhaps not unc 
they may be termed “superstitions.” 

Although the biological utility © 
reinforcement. is obvious. certain. dilemmas 
“effects” follow a given action and are un- 


arises. the problem of “temporal 
means of symbols (Mowrer, 
| this connection is severely 


f learning where there is delayed 
nevertheless arise where 


two reinforcements Or 
A in time. Here 
1 best solves by 
lower animals it 


c enly distributed 
integration,” which mar 


1960); the capacity of 
limited. 

e that occurs quickly than 
“intelligent,” but slower- 
is just 


ea protective responsi 


to hav 
ating, more 


a more discrimin 
"he "trouble" w ith the short-latency rertexe 


f occurrence, they sacrifice integration with the 
anisms which function autonomously, 
organized system Ww hich we call the 


eyes, it is more important 
one under the control of 
acting (“deliberate”) system. 
this; thar in achieving speed © 
rest of the organism. ‘They are little mec 
independently of the larger, more highly 


LS ] ae "hapters 6 and 7). 
organism-as-a-whole (ef. Mowrcer. 1960, unn uns 2 waa i 

‘fe : ; ASIE ies cations involve os 
Kor a good discussior f some of the complications involved in research on 


diis prabiene, sé cheer ( also Armus (1959)+ Karsh (1959), and Moll (1959) - 


10 
1956); 


Il 


Unlearning, Conflict, 
Frustration, and Courage 


In the early chapters of this book, 
toward the “negative” aspects of learr 
active and passive avoidance beh 
more systematic underst 
ourselves exploring the problem of secondary 
logically divides itself into type-1 and type-2. Then, upon further 
analysis of type-2 secondary reinforcement, we found we were 
position to formulate a new theory of * 
behavior and habit. In the immedi 
has fallen rathe 
of learning 


attention was especially directed 
ning, i.c., the learning involved in 
avior. In seeking to gain a better, 
anding of these two phenomena, we found 
reinforcement, which 


in a 
"positive" learning, i.c., approach 
ately preceding ch 
r evenly upon both the positive 
+ as thus conceived. Now the 
problem of Unlearning, 
"direction," c 


apters, the emphasis 
and the negative aspects 
time has come to examine the 
that is, the questi 
an be undone, counteracted, 
At once a par: 


on of how learning, in either 
reversed. 
adox will be encountered: If the 
forms of "reinforcement," 


crement respectively, and 
what is "positive" o 


TC are two distinct 
drive decrement and drive in- 
antithetical in their effects, then 
movement with respect to one of 
will involve a reversal of the other. Thus, 
one sense negative is in another sense positive 
(cf. Deese, 1952, Chapter 6; Keller, 1954 pp. 31 ff.). One of the im- 
portant challenges to revised two-factor theory is to scc 
can handle this complication Land the i 
conflict. 


involving 
if these are 


r “forward” 
these forms of learning 


learning which is in 


how well it 
attendant phenomenon of 


It is obvious that unlcarning of both hopes 


and fears can be pro- 
388 
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duced bv counterconditioning; but it is also well established that 


these reactions recede under the less radical influence of “extinction,” 
ic, elicitation without confirmation. Discussion of the latter. phe- 
"frustration"; and from consideration 


nomenon leads to the topic of 
av emerging which 


thereof it is found that a theory of extinction is tod 
is consistent with the general view of unlearning adopted in this 
chapter. 

In this context we shall examine a p! 
has only recently 


venomenon which, while equally 
begun to be an object 
of "courage." As we 
and readily lends it- 


familiar to common sense, 
of laboratory study. This is the phenomenon 
shall see, it, too, is easily defined operationally 


self to objective investigation. 


l. The Problem of Unlearning in Light of 
Revised Two-Factor Theory 

] version of Thorndike's Law 
confronted the problem of unlearning and had 
it held, reverses. obliterates, “stamps out” 
Learning, starting at or near 
of reward; and under the 
TCOTCSSCS, back toward its starting 
V of accounting for the conflict 
did it take cognizance of 


apter 2. the origina 


n indicated in C 
of Effect squarely 
àn explanation: punishment, 
the neurological residue of past learning. 
"zero," progresses under the influence 
impact. of. punishment, it simply 


point. But this approach had no way 
estly produces; nor 
not only interferes ith established behavior 
poi) but also can set up behavior 
was no behavior, no 


Which punishment manif 
the fact that “punishment” 
(“passive avoidance’; see 
(active avoidance"; sec Chapter 
action before. One of the major assets O 
as formerly conceived W owledged and 
both of these phenomena. of Effect had too little con- 
flict, this version of two-factory theor one may say, 100 much. 

The dilemma to which the carlicr (second ) version of two-factor 


theory led may be summarized as follows. It accepted. the "Thorn- 
: connection between à drive 


Chapter 
3) where there 
f two-factor learning theory 


as that it ackn accommodated 


But if the Law 
Y had. 


dikian hypothesis that the neurological 
: ntal response 15 str 
it rejected the complementary notion that st 


ened by punishment. It proposed instead th: 
stimuli produced by the behav ior which punishment follows to become 


conditioned to fears and ] that. behavior instigated 
by fear and reinforced. by competes with and 


often overrides the original 


cnethened by rew ard, but 
ich a connection is W cak- 
n punishment causes the 


stimulus and an instrume 


it further posite 
fear reduction then 


behavior. This way of thinking made a 
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place for the phenomenon of conflict, as the Lai of pana M S 
but it provided no suggestion as to how conflict T even ^ ne 
resolved. One learning mechanism, that of motivation and reward, 
caused a "habit" to be established; punishment, through pu ddl 
ing, then caused a new motive, that of fear, to come T egy 
and its reduction served to reinforce an antagonistic “habit.” But such 
antagonism or conflict, once created, might be expected to hold the 
subject in a sort of rigor or tetany indefinitely —unless some sort of 
central short-circuiting or, perhaps, “stamping out" of the kind posited 
by Thorndike mercifully intervened. The question is now: Does re- 
vised two-factor theory 'offer any simpler or more satisfactory way of 
dealing with this problem? 

The principal difference between the second version and the present 
revision of two-factor theory is that, in the latter, 
made dependent upon conditioning no less than 
In the carlier version, the learnin 


habit formation is 
is habit. inhibition. 
g produced by reward was thought 
of as involving a strengthening of the 

which leads to the reward; whereas the 
was conccived, 


stimulus-response sequence 
effect produced by punishment 
not as simply reversing the effects 
but as leading to the conditioning of fe: 
occurrence of the stimulus-response sequence which elicited the pun- 
ishment. The unsatisfactory nature of primitive h 

sented by the Law of Effect, first bee 


punishment and avoidance behavior and led to the earlier two-f 
position. Now, for reasons previously reviewed 
tioning is responsible for the effects of reward no less than for the 
effects of punishment. Hence, all learning is reduced to sign learning 
(with “feedback” control of behavior; see Chapter 7). By virtue of this 
new and more consistent position, the problem of unlearning and of 
conflict is manifestly changed. 

Let us begin by reviewing 
ing learning and unle 
volved. We know th 


of prior reward, 
ar to stimuli associated with the 


abit theory, as repre- 
ame evident from the study of 


actor 
s it seems that condi- 


some well-authenticated f. 
arning when independent, external 
at if such a stimulus, as 
with one type of reinforcement, is given a particular meaning, this 
meaning can be changed cither by (a) repeated presentation uf the 
stimulus without such reinforcement or by (b) association with the 
Opposite type of reinforcement. For example, if a given stimulus, 
through association with the onset of pain, has first 

capacity to arouse fear, its “meaning” can be changed, 
of neutrality, by the repetitive occur h 
the pain; or the stimulus can be c 


acts concern- 
signs are in- 
a result of association 


acquired. the 
in the direction 
renee of the stimulus arhont 


aused to lose its fcar-arousing tend- 
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eney and take on hope-arousing capacity by its repeated conjunction 
with a pleasurable experience of some sort. The first of these pro- 
cedures is ordinarily referred to as extinction and the other as counter- 
conditioning. i 

, We may suppose, then, that the 
is controlled by these same principles, except 
response-correl 


selection or elimination of behavior 
that here the condition- 


ine inv M nt " 
ag involves stimuli that are ated rather than inde- 
pendent. A response which has previously had little or no “habit 
occur in a given situation if 


stre ^ i 
trength" may be made more likely to 
associated therewith become 


follow ed by rew ard, since the stimuli 
conditioned to hope; and once established in this way, the same be- 
havior can be nullified cither by elimination of the reward or, more 
rd with punishment, so that the stimuli 
rather than hope or positive expecta- 
ng changes in the 


swiftly : 
wiftly, by replacing the rewa 
associated therewith arouse fear 
tion, Here no assumption whatever is made concernir 


conductance or resistance of the neural connections between the or- 
- be called the "executive" 


rather in the nature of 


ganism’s brain and muscles, ie. in what ma 
(“ordering”) pathways. The change occurs 
the “information” relaved back to the brain when a given course of 
action is started or contemplated (ie, is symbolically rehearsed; 
Mowrer, 1960, Chapters 2 and 5). It is assumed that if response- 


Produced stimuli become conditioned to fear, the likelihood of recur- 
be decreased and that if response-produced 


c, the likelihood of recurrence will 
forcement, both types of condi- 
“indifference” point, with a 
response inhibition, as 


IEG of that response will 
stimuli become conditioned to hop 


be increased and that, with nonrein 


tioning will extinguish, toward a null or 


concomitant lessening in response strength or 


the case may be. 

"Thus, if we assume that 
Cue-response bond but a matter 
associated with the occurrence 
account for loss of habit stre! 
countercondit 
tor conccptior 


is ncither a motive-response nor a 
of hope being conditioned to stimuli 
of a particular response. it is casy to 
gth through either extinction. (non- 
tioning (opposing reinforcement). If 
a of habit formation is valid and if 
nal signs can be modified in the wavs indicated, 
the same principles. behavior itself can be 
meaning of behavior-connected stimuli. 
shall find that habit is the same as condition- 
conditioning: the conditioning of hopes to 
dependent, stimuli. 


“habi 


reinforcement) or 
the revised. rwo-fac 
the meaning of exter 
then it follows that, by 
Modified by changing the 

Here, as elsewhere, We 


ing—that, in fact, it /5 
rather than 1n 


Tesponse-ct related, T 
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Il. Conditioning and Counterconditioning 
with External Stimuli 


In the preceding discussion it has been taken for granted that id 
affective responses made to external stimuli can be changed decüru- 
ing to definite and orderly principles. Before going further it will be 
useful to examine some of the relevant laboratory evidence for such 
an assumption. Oddly enough, one of the earliest 
this kind was published in 1920 by John Watson, the great advocate 
of Behaviorism, in collaboration with Rosalie Rayner. The introduc- 
tory paragraph of their report reads as follows: 


and best studies. of 


In recent literature various speculations have been entered 
the possibility of conditioning v 


experimental evidence in supp 


into concerning 
"arious types of emotional response, but direct 
ort of such a view 


g atterns are few 
of fear, rage and love, then there must be 

which the range of stimuli which can call out these emotions and their com- 
pounds is greatly increased. Otherw ise, complexity in 
not be accounted for. These authors without adequate experimental evidence 
advanced the view that this range was increased by means of conditioned 
reflex factors. It was suggested there that the carly home life of the child 
furnishes laboratory situation for establishing conditioned emotional re- 


sponses. The present authors have recently put the whole matter to an 
experiment test (p. I) 


has been lac king. If the 
theory advanced by Watson and Morgan (1917) to the effect that in infanc 
the original emotional reaction p 


» consisting so far as observet 
some simple method by means of 


adult response could 


There then follows an 
experiments with the 
almost from birth in 
nurse in the Harriet | 
scribed as an unusu 


account of the now-classical W 
year-old infant, “Albert Bat 
a hospital environment; his 


atson-Rayner 
He was “reared 


mother was a wet 
ane Home for Invalid Children, 


ally healthy and unemotional child, 
stability was one of the principal reasons for using 


this test" (p. 1). First of all, Albert M 
less “furry” objects: “a w hite rat, a . à monkey, masks 
with and w ithout hair, cotton wool, burning newspapers, etc. . . 
At no time did this infant ever show fear. aP (p. 2) 

The plan was to select the white rat as a “conditioned stimulus" 
and see if it could be made to elicit siens of fear by Associating it with 
TO g y associating 
the noise produced by striking a a sharp blow." ‘The in- 
vestigators wished to Obtain, answers to the 

questions: 


` He was dce- 
In fact, “his 


him as a subject in 
as shown an array: 


rabbit, a dog 


of more or 


"steel bar 


1f possible, following four 
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Can we condition fear of an animal, ¢.g.. à white rat, by visually pre- 
iking a steel bar? 
al response be established, will there be a 


senting it and simultaneously stri 
ea at conditioned emotional | o 
to other animals or other objects? 
MI. What is the effect of time upon suc 
sponses? 


h conditioned emotional re- 


eriod such emotional responses have not died 


IV. If after a reasonable p 
devised for their removal? (p. 3)- 


oO 
ut, what laboratory methods can be 
] in this investigation are 


he first two conditioning trials employee 


then described as follows: 

and presented to. Albert. He 
his hand touched. the animal 
1. The infant jumped violently 


i White rar suddenly taken from the basket 
T i ` : à 
in gan to reach for rat with left hand. Just as 
: 3E bar was struck immediately bevond his head 
and fell forward, burying his face in the mattress. He did not cry, however. 
Just as the right hand touched the rat fon the second trial] the bar was 
again struck. Again the infant jumped violently, fell forward and began to 
whimper (p. 4 
One week later there were five more “joint stimulations.” On the 
Cighth trial, the rat alone was presented. 
by began to cry. Almost instantly he 
left side, raised himself on all fours 
he with difficulty before 


vas shown the ba 
fell over on 
diy that he was caug 


The instant the rat v 
turned sharply to the left, 
and began to crawl away 50 rapi 
Teaching the end of the table. ^ 

This was as convincing 3 case ofa completely conditione ; : 
could have been theoretically pictured. In all, seven joint stimulations were 
given to bring about the complete reaction. It is not unlikely had the sound 
been of greater intensity oF of a more complex clang character that the 
number of joint stimulations 1 


d fear response as 


night have been materially reduced (p. 5)- 


Five days later Albert was tested. for generalization of fear to other 
objects. Wooden blocks were accepted and m 


then: 


anipulated as usual. But 


sess in front of him. The rë- 
ned as far 
rs (p- 6). 


on the matt 
gan at once. He le: 


hen burst into tea 


aly placed 
ative 


Fhe rabbit was sudder 
ac 
ction was pronounced. Ne 
away from the animal as possible, 


responses be 
w himpered, t 


generalization were like- 


reactions to à dog. a fur coat, cotton 


Less marked but definite evidences of fear 


Wise observed in the little boy 5 Ans 
Wool, and a Santa Claus mask. He. was also completely negative 
rimenters "put his head down to see if Albert 


+). There is thus clear-cut evidence 
at similar to the white rat and 
wooden blocks. 


When one of the expe 
would play with his hair” (P. 
of transfer of fear tO objects somewh 
of nó transfer to more dissimilar objects. C£ 
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The experimenters were also able to show that when Sisi a 
placed in a room other than that in which original Sanches | = 
occurred, the rat and other “furry” objects produced some kw 
drawal and fear tendencies, but they were less marked than they fee 
been in the other situation. Thus it was demonstrated that aie 
is there a gradient of fear generalization from the original conditione 
stimulus (white rat) to other similar objects but that there is also a 
gradient of generalization in the fear reactions shown by the subject 
tó these several objects from the original learning situation to a dif- 
ferent, neutral situation. 

What, then, of the “effect of time upon conditioned emotional re- 
sponses”? (p. 10). As a result of tests made after a | 


the investigators sa y: 


apse of one month, 


These experiments would seem to show conclusively that directly. con- 
ditioned emotional responses as well as those conditioned by transfer persist, 


although with a certain loss in the intensity of the reaction, for a longer 
period than one month (p. 12). 


Because of the unavoidable removal of Albe 
setting, Watson and Rayner were unable 
which might have climinated his fe 
jects. Had he been available for 


rt from the hospital 
to experiment with methods 
ar of the white rat and similar ob- 
a longer period, they would have 
attempted “reconditioning” by “feeding the subject candy or other 
food just as the animal is shown” (p. 12) and by providing other 
pleasant experiences in this context, 


An incidental observation made by these workers which is not 
without interest and relevance here is that, although not a "thumb- 
sucker" under ordinary circumstances, 
his thumb in his mouth * 
and noxious stimuli." 


Albert was very prone to put 
‘as a compensatory dev 


¢ for blocking fear 


During the course of these ex 
noticed that whenever Albert w 
generally he would continually 
ment the hand reached the mout 
ducing fear (p. 13). 


periments, especially 
as on the verge of te 
thrust his thumb in 
h he bee 


in the final test, it was 
ars or emotionally upset 
to his mouth. The mo- 
ame impervious to the stimuli pro- 


This type of observation h 


as been re 
situation; and it squ 


ares well with our 
are antithetical and that, 


will counteract the other, 


peated in many other types of 
assumption that fear and hope 
on Which is the stronger, one 


It may be that Freud (1916) is correct in 
his contention that the mouth is an “instinctively” 


and that stimulation thereof produces 


depending up 


erogenous zone 
à pleasurable sensation no less 
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reflexly and innately than does stimulation of the sex organs (cf. the 
of certain tastes, Chapter 6). 


discussion of the innate “palatability” 
oral stimulation regularly pre- 


But it is also conceivable that since 
cedes hunger (and. hunger-fear) reduction, it acquires quite powerful 
and, in this wav, becomes capable 


secondary-reinforcement properties 
Albert was not especially, 


of neutralizing fear. So far as we know, 
ere conducted with him, so it may 
ave sought, under these condi- 


ociated with cat- 


if at all, hungry when the tests « 


at first not scem reasonable for him to h 
ation. which had been a: 
ay laboratory demonstrations of the 


the tendency for living 


tions, to re-activate stimul 


ing. But there are now a good nm: 
reinforcement, İ.C., 
have been associated with one type 
of fear reduction, to attempt to reproduce these same stimuli when 
experiencing fear of a different type. “Anxious cating” has already 
been alluded to, and “anxious sucking" would seem to be a closely re- 
lated phenomenon (sce Chapter 3). 

In view of the inability of Watson and Rayner to study fear de- 
it is fortunate that Mary Cover Jones, a few 
rested in this problem and carried out a num- 
Instead of setting up specific fears experi- 
signed to "undo" them, 
more difficult task of 


generalization of secondary 
organisms, after certain stimuli 


conditioning in Albert, 


vears later, became inter 
ber of illuminating studies. 
mentally and then introduci 
Dr. Jones (19242) undertook ' 
climinating fears which children had already acquired in their daily 
3$ ds—disuse, verbal appeal, negative adaptation, re- 
and social imitation—were tested for their ef- 
tion; but the approach which worked best 
" je was described thus: 


ng conditions de 
the presumably 


lives. Various metho 
pression,! distraction, 
fectiveness in this connec 
Was that of “direct conditioning. 
od, the child is placed ina high chair and 
The fear-object is brought in, starting à negative re- 
sponse. It is then moved away gradually until it is at a sufficient distance not 
to interfere with the child's cating. The relative strength of the fear impulse 
and the hunger impulse may be gauged by the distance to Ww hich it is nieces- 
sary tó remove che fear object. While the child is cating, the object is slowly 
brought neeo the table, then placed upon the table, and finally as the 
tolerance increases it is brought close enough to be touched. . . . The cffec- 
tiveness of this method increases greatly as the hunger grows, at least up to 
a certain point. The case of Peter | reported in detail elsewhere, Jones, 1924b] 
dure Jones implies by the word "repression," 
fear response. We have some understand- 
esponse-correlated stimuli are fol- 


aving for fe 


: During a period of cr 
given something to eat. 


xactly what proce 


diy 
It is not clear € 
brent of the 


but it at least suggests pus 


ing of what to expect when independent or P i 
lowed by drive increments do not know, in any systematic: waya what 


but we 
itself is thus reinforced. The problem manifestly needs further 


happens when fear 
attention (see Section VD- 
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illustrates our procedure, one of our most serious problem cases, he Was 
treated by the method daily or twice daily for a period of two months. The 
laboratory notes for the first and the last days of the training period show an 
improvement which we were able to attribute specifically to the training 
measures used. 


Case 30.—Peter. Age 2 years, 10 months. ; : : l . 

March 10, 10:15 a.m. Peter sitting in high chair, c: ing candy. I Nperimenter 
entered room with a rabbit in an open mesh wire cage. The rabbit w as placed 
on the table 4 feet from Peter who immediately began to cry, insisting that 
the rabbit be taken awav. Continued crving until the rabbit was put down 
20 feet away. He then started again on the candy, but continued to fuss, “I 
want you to put Bunny outside.” After three minutes he once more burst into 
tears; the rabbit was removed. 

April 29, 9:55 a.m. Peter standing in high ch 
He inquired, “Where is the rabbit?” 


air, looking out of the window. 


"The rabbit was put down on the chair at 
Peter's feet. Peter patted him, tried to pick him up, but finding the rabbit 


too heavy asked the experimenter to help in lifting him to the window sill, 
where he plaved with him for several minutes, 
This method obviously requires delicate handling. Two response systems 
are being dealt with: food leading to a positive rea ti 
ing to a negative reaction. The 
forming the fear-object into a « 
lus). But a careless manipulator 
taching a fear reaction to the 


ction, and fear-object lead- 
desired conditioning should result in trans- 
urce of positive response (substitute stimu- 
could readily produce the reverse result, at- 
sight of food (pp. 388-389). 
By way of summarizing her 


work, Dr. Jones says, in part: 
In our study of methods for removing fear responses, we 
fied success with only two. By the method of dir 
ated the fear-object with a crav 
response. By the method of soc 


under controlled conditions, th 
cially chosen with a view 


found unquali- 
1 ct conditioning we associ- 
ing-object, and replaced the fear by a positive 
ial imitation we allowed the subject to share, 
c social activity of a group of children espe- 
to prestige effect, , . (p. 390). 
In the more detailed report of Peter's "recovery" already cited ( Jones, 
1924b), the process of reconditioning is shown quantitatively: by 
means of the graph reproduced in Fig. tai along 
the ordinate of this figure represent a range in toleration of the 
originally fear-producing object from “A, Rabbit anywhere in the 
room in a cage causes fear reactions” to «p. Fondles rabbit affec- 
. A. m " : R , Nn ^ P 
tionatelv" and “Q, Lets rabbit nibble his fingers." The graph there- 
fore shows a progressive and decided change in Peter's attitude toward 
the rabbit, from strong fear to positive interest and affection 

Ihe work of Watson & Rayner and of Jones thus demonstrates 
that by association with a NOXIOUS experience an originally neutral (or 

Mantel E 


positive) object can become 4 potent fear- 
producer and that bv association with a pleasurable experience an 


The letters shown along 


even an emotionally 
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originally neutral, or even an emotionally negative, object can be 
ositive and pleasant connotations. Thus 
"cement. with Thorndike in his contention that learning 
` but we depart from his position in two important ways. 
of learning is not in the S—R bond 


made to take on emotionally p 
we are in $ 


is “reversible, 

We assume (1) that the "locus" 
s 

but in the Rig Se connections. And we 
s 

move in cither direction on à 

lope) through o.o (indifference, neutrality) to 

rather than merely back and forth on a scale extending from 0.0 (no 

) +1.0 (maximal habit strength). 

from these distinctions have 


assume (2) that learning may 


“scale” extending from + 1.0 (strong 
; —1o (strong fear), 


abit strength, no "bond") to 

Many of the implications which flow 
one may 
(of which Hull's i 
ase can have is zero 


already been noted; bur a new be appropriately suggested 
"Thorndike's scheme ( a variant), 
a given respor 
words, be less than zero in this 
But in point of fact, we 


a dog which has been 


dere. According to 
the “least” habit strength which 
(0.0). Habit strength cannot. in other 


system, ic. it cannot have a negatis 
easily exist: €g- 


value. 


know that such a state can 


—-cxzr-zzovo 


»goomoO-o 


10 15 20 25 30 35 40 45 
Test days 
roduced from Mary Cover Jones (1924b, p. 311) showing 
a fear response in a small boy. 
aate is given in the text. The lower-case letters aC- 
mate points at W hich vicissitudes in the daily 
imental situation occurred and facilitated or re- 


ri 
aditioning process. These are described in some 


Fig. 11-1. A graph rep ds ) 
the progressive extinction of The meaning of the 
capital letters along the ordir 
Companying the curve 
life of the child or in the expe 
and recor 


itself de 


tarded the extinction 
detail in the original report 
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everely shocked for raising his right forepaw (or after having it 
" : x 2 ini E E ^re- 
assively lifted) as, let us sav, a means of obtaining food, will there 
enin not only not make this response—he will actively re 


ist (“push 
against”) this response, both when the response is instigated se 
(cf. Konorski’s experiments, Section X, Chapter 8) and w hen np : : 
tempt is made by the experimenter to “perform the oen ^ inve 
dog. Negative habit strength, or response resistance, which we gna 
to be an empirical reality, is readily derivable from revised two-factol 
theory, but it is not deducible from the Thorndikian theory of learn- 
ing. In addition, neither can the latter give an explanation. of place 
learning (approach and avoidance), whereas place learning is no less 
casily accounted for in the present system. (C apter 9) than are re- 
sponse facilitation and active response inhibition. 


IIl. Nonconfirmation versus 
Counterconditioning: the Fatigue Hypothesis 


In the preceding section we have seen some instances of emotional 


conditioning and counterconditioning to external stimuli. Remaining 
is the question whether so-called extinction is si 


ing which is carried just to the null point (of no meaning) or involves 
rather a different process of some sort, 

In the second version of two-factor 
extinction. constituted something of an enigma. 
“habits,” instead of being intrinsicall k 
merely blocked by the competing motive of fear, 
interpret the loss of habit strength which occurs w 


mere nonrewarded repetition? In the introduction of à republished 
version of an early paper by Mowrer 


& Jones (Mowrer, 19502), this 
problem has been stated—and ostensibly solved— 


mply countercondition- 


theory, loss of learning through 
If it is assumed. that 
y weakened by punishment, are 
then how is one to 
ith mere extinction, 


as follows: 
Any careful investigation of the 
of the inhibition of behavior that is produced by Means of so-called “punish- 
ment.” Other writers | notably Thorndike| have suggested that this effect 
comes about by virtue of “punishment” hay ing the capacity to neutralize or 
reverse the effects of previous experienced "req ard." It now seeing that in- 
hibition occurs rather as a result of conflict, which consists of the incompati- 
bility of the punished response and the response(s) which the punishment it- 
self and the associated fear call forth. (What is said here is intended to apply 
only to the inhibition of overt behavior, How emotional reactions are nega- 
tively modified is an independent and, at present, not well understood prob- 


lem.) | This comment was made in the context of unrevised two-factor learn- 
ing theory (see Section VI ).| 


In this paper 


psychology: of fear leads to a consideration 


the same logic is extended to account for the inhibition of 
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e ordinary sense, but by what is often referred 
ce of conflict is that 
wewarded perform- 


habits, not by punishment in th 
to as "extinction." In extinction the only apparent sour 
provided by the fatigue resulting from the recurrent, ur 
ance of a habit. The experiment here reported was designed to test the de- 
duction that, with other things equal, effortful responses, ie, responses 
latively great fatigue, will extinguish (in- 
) his inference was confirmed at 


Whose performance generates r 
hibit) more quickly than less effortful ones. T 
a high level of confidence (p. 152)- 

t forward might be reformulated, somewhat 
that. both punished and nonrewarded 
ally similar reasons. In the one case it 
is as if the subject says to himself, SE perform that response 1 will 
get burt. In the other case it is as if the subject says to himself, “If I 
perform that response ] will get tired.” Both pain and fatigue are forms 
red that they both operate, the one swiftly, 
inhibit any response which produces 


Phe hypothesis thus pu 
more colloquially, by saving 
responses are inhibited for basic 


of discomfort; and it is assum 
the other more gradually, to 


them. 
d to in the paragraph last quoted 


wis. referre 
. During prior training 


form, in Fig. 11- 
Į learned to press 3 Skinner bar in order 
at different stages of training, 
The subjects were then 


The experimental findir 
are reproduced, in graphic 
30 hungry laboratory rats hac 
to obtain food, with the bar requiring, 
§, 42.5, and 8o grams of pressure te depress it. 


Extinction responses 


15 20 


piri 
15 20 0 
n Minutes, day 2 Minutes, day 3 


oLa 

0 5 
Minutes. day 1 

Mf. extinction of a habit w hen that habit is 


ng the course ¢ 
£ (I) with respect to the effort required to 


Fig. 11-2. Curves show! r 
easy (1), hard (Hb. and intermediate 
execute it (Mowrer and Jones, 1943. p 37477 
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i n H exti zti ‘ AU "ried 
randomly divided into three equal groups; and extinction was carr 


out with the bar so weighted as to require one of the 


threc pressures 


i in Fi 3 re was a decided (and 
just listed. As will be seen in Fig. 11-3, there was a decided ( 
18^ SER Po a REEL i ER [1117 
503 É 
350 |À 
300 
v E 
2 gen Fig. 11-3. Average number o 
E i is " 
5 extinction responses made by 
3 " Py 
© 200 three different groups of ani- 
5 mals as à function of response 
© 150 effortfulness (Mowrer and 
Š 100 1107 Jones, 1943, pP- 375). 
50 
(oy MA EE es a i e) 
05 42.5 80 


Grams 


of extinction. and 


pressing. In. sum- 
ation Mowrer & Jones say: 


statistically reliable) relationship between rapidity 
the effort involved in performing the act of bar 
marizing this part of their investiga 


This study takes as its point of departure the hypothesis th 

tion of a response, or habit, through its own nonrewarded repetition involv 

a conflict in which the fatigue thus generated instigates a response (resting) 
which is incompatible with and therefore tends to inhibit the original re- 
sponse. Such an hypothesis is confirmed by the finding that the rate at which 
a nonrewarded response c tinguishes is highly correlated with the effortful- 
ness of that response. It is also confirmed by the well-known tendency for an 
extinguished response to reappear after a lapse of time, since the motive 


which produces the inhibitory response of resting, namely, fatigue, is thereby 
eliminated. The fact that the amount of recovery following succ 

tions becomes progressively smaller and may eventually reach zero is due 
to a change in the "meaning" of both situational and organic cues, which 
follow familiar learning principles (p. 169). M 


at the climina- 


successive extinc- 


Using the empirical findings here 
studies by Solomon, 19482, 1948b 
for his reasoning, Hull (1943) introduced the 
tion" (Ir) and "conditioned inhibition" 
to the two phenomena described in the f 
(1949) has paraphrased Hull's definition 


reported (see also 


confirmatory 
. and Applezweig, 


1951) as the basis 
terms, “reactive inhibi- 
(sli). to refer, respectively, 
oregoing paragraph. Kimble 
of reactive inhibition thus: 


UNLEARNING, CONFLICT, FRUSTRATION, AND COURAGE 401 


cactive inhibition is essentially a negative drive state closely allied to pain. 


8T xix nataran prodiaet Te which er from sr accompanies 
of rest. In this respect ly resembles riim vut 2 a ee tea gu 
: g p- 15) 

If nonrewarded repetition of a response is to have anything more 
than a temporarily depressing influence, it is clear that something must 
happen comparable to the fear learning involved in the use of punish- 
ment, It was therefore assumed, by Mowrer & Jones, that some part 
of the “reactive inhibition,” or farigue reaction, produced by non- 
rewarded repetition could become conditioned to stimuli associated 
response in question, so that subsequently 
the response could be immediately inhibited, instead of having to occur 
rate full-fledged fatigue. 
inhibition through extinction. was 
learning theory and in 
lv be the 


with the occurrence of the 


enough times to gener 

This, then, is how the matter of 
handled in the second version of two-factor 
Hull's formal theoretical system. However, this can scarce 
end of the story. In the first place. t Il some question about thc 
empirical validity of the ssumption that effortful responses extinguish 
More rapidly than casy ones. Such an assumption certainly agrees with 


common observation; and Solomon and Applezweig. as already noted, 
laboratory findings. However Maatsch, 


paper entitled "Effort and resistance 
report data which thev 
In their study these writers used 
oved by Mowrer & Jones. 
vere lacking in the earlier 
of which the authors 


here is sti 


have reported confirmatory 
Adelman, & Denny (1954). iP 2 
to extinction. of the bar-pressing 
assumption. 
lar to that empl 


but with certain controls W hich they felt we 
produced in Fig. 1174 


response," 


feel go against such an 
an over-all procedure simi 


study. Their findings arc ri 
say: 2 

within the limits employed, bears 
o extinction, and as a consequence, to the ac- 
ained by Mowrer and Jones in a comparable 
differences in habit strength produced by 


es that work, 


The present study indi 
No relationship to resistance ti 
cumulation of I. The data obt 
situation may be attributed to the 
their preliminary training schedule. -++ hie 
, The Mowrer and Jones study, far from being convincing evidence of a rela- 
Honship between work and the accumulation of li would seem to support 


the lack of relationship found in the present study. 


ccount 


“reactive inhibition" to 4 
never used the term “punishment.” A rare exception 
Hull, Livingston. Rouse, & Barker (1951). where he 
t) as distinguished from experi- 
hapter 6, Section VD. 


* Hull frequently referred. to the concept. of 
for extinction, but almost 
will be found on p. 44 of 
speaks of "negative reinforcement 

ment extinction". (Chapter 3- Section Ill, and C 


( punishmen 
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S imi T vere reporte av ontgomery 
x E 2 , x rej v Monte \ 
onflicting results of a similar nature v 
Cc g 
OE ded, in light of the present f available literature, that 
: It is fies d, in ligl Pul present study and available i et ARE 
explan tions « i i 3 a based upon the effects of lesser a D 
xpl: s of extinction phenomena based I T “ts E 
i wer 5 Ed ig c enc ered in Ski sr boxes, T mazes, and othe 
such as ht be encountered in winner boxes E 4 ; 
of work, such as mig ) j S ; : ver 
common learning situations, are questionable and contingent upon furthe 
$ 
rescarch in this arca (p- 49). 
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Fig. 11-4, Response extinction 


as 
& Denny, 1954. p- 49). 


à function of response effort CMaar eh, Adelman, 


Whether these criticisms of the 
be supported or refuted by further empirica 
tion (see Aiken, 1957; Lawson & Brownstein 
1954; Capehart, Viney, & Hulicka, 195 
But there is another issue which has 
raised by Maatsch ez al. In 
plicable only to 
musculature, i.c., 


fatigue theory of extinction will 
l inquiry is an open ques- 
+ 49375 Stanley & Aamodt, 
35; and Chapter >, Section VI). 
logical precedence over the onc 
any case, the fatigue hypothesis is ap- 

the extinction of responses involving the skeletal 
to “habits.” The fact is, of course, that purely ius 

nomic responses, such anic skin reaction (fear) or salivation. 
epeatedly. elicited without reinforcement; 
a question here of 
ary sense of the term, Only 
involving skeletal (striated ) l 
as a consciously expe 
mediated by the 


as the galv 
also undergo extinction if r 
and there can searcely be "fatigue," at least not in 
any ordin " 


in connection with responses 
muscles do we 


normally speak of fatigue 
rienced countermotiy 


ation. Emotional responses. 
autonomic nervous Syste 


smooth muscle, can manifestly be 
their repeated elicitation does not, 
thing like fatigue. He 


m and involving glands and 
conditioned and deconditioned, yet 
so far 


as we know, gener: 
nee we must have 


c any- 
something other than a fatigue 
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theory to account for the extinction of emotional responses. The 
as described above, is evidently a form of counter- 
"pure form." The fact seems 
al response (notably 


fatigue theory, 
conditioning, rather than extinction in 
to be that repeated nonconfirmation of an emotion 
hope or fear) also tends to cause that response to disappear. Its dis- 
explicit counterconditioning; but, as 
already noted, mere nonconfirmation seems to be a sufficient con- 
dition for the gradual elimination of such responses." The question is: 
" a subtle form of counterconditioning or 
This problem will be given 


appearance can be hastened by 


Does such a procedure involve 
some ditferent principle or mechanism? 
special consideration in the next section. 


IV. Extinction and the 
Concept of Frustration 


atedly seen how unsatisfactory wa the 


In this volume we have repe A 5 pasi T 
treatment of punishment and avoidance learning until the subjective 
phenomenon of fear was acknowledged and systematically related to 


theory; and we have also followed the growth of interest in so-called 


ent, which seems to ber 
Contemporary learning theory, with 
ant to acknowledge the reality 


secondary reinforcem nothing more nor less than 
the common experience of bope. 
its behavioristic bias, has also been rcluct 
E cas selactance has Jowly overcome, and this 
of ager; bur that reluctance has been slowly 
This discussion is given special point by an experiment recently reported. by 
Black (igs), which "consisted. of teaching dogs an acuve avoidance response 
tunder threat of electric shock) 3 ad then repeatedly presenung the danger signal 
while the subjects were muscularly paralyzed (by curare) but 
"Dogs receiving extinction. trials while immobilized 
by d-tubocurarine chloride required fewer trials to reach the extinction criterion 
and made fewer responses during extinction (when tested after recovering from 
curarization) than animals W hich had not received extinction trials under curare 
(p. 524). And earlier Solomon, Kamin, & Wynne (1953). showed that à similar 
effect ea be produced bv means of a glass barrier, W hich prevents dogs from 


executing a shuttling response and thus forces them to "reality test” (during ex- 
ym er that. the shock "isn't there” anymore, Hence, on sound 


(sec. also Hurwitz, 1955. and Amsel, 958b), we can say 
of an overt response is not essential for its extinction 
it can hardly be argued that the fatigue or effort 
However, the fact of emotional ex- 
still 


(without shock) 
fully conscious and sensate- 


tinction) and discove 
experimental grounds 
that since the performance 
(see. Mowrer, 1960, Chapter 


factor is the sole determiner 
ht does not exc 


Mowrer & Jones. 
yet find the phenomenon of extinction, in all 


master formula (see Section IV). 


of extinction. 
lude the possibility: that effort ma 


tinction in its own rig 
operate as suggested by 
but we may 
d under a single 


To coin a phrase, the situation seems 


somewhat complex: 
its guises, can be subsume 
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emotion, like the others Set pradually finding its rightful place in 
scientific analysis of behavior. | 

RU Doob, Miller, Mowrer, & Sears made the first e 
tended effort to deal with this problem in their book, F ead i 7 
Aggression. Despite some useful features, this effort was unt VIS unt 
cessful: It drew much of its inspiration from psychoanalytic gee 
lations concerning aggression and hostility, as viewed clinically; anid 
it was at the same time dominated, methodologically, by a strong 
behavioristic slant which required that frustration and aggression be 
treated in a simple stimulus-response framework, which had little 
or no place for the intervening variable of anger (cf. Miller & Steven- 
son, 1936; also Lewin, 1956). A 

In 1951 Brown & Farber made a more realistic attack upon this prob- 
lem in an article entitled “Emotions Conceptualized as Intervening 
Variables—with Suggestions Toward a Theory of Frustration.” Heart- 
ened by the successful introduction of fear as an intervening variable 
(and by the emerging recognition of hope), these writers argued, per- 
suasively enough, that the time had col 
anger in the context of systematic learning theory. Since Hull's trcat- 
ment of the latter was the most highly formal and claborate then 
available, these writers used it as the matrix for their postulations con- 
cerning the role of anger. That system, as we now know, has a number 
of major (if not fatal) weaknesses: so no attempt will be made here to 
show the ways in which Brown & Farber related their analysis to it. 

One of the many more fortunate features of th 
writers point out the inherent. rel 
(trial-and-error learning) and the ex 
ofa blocking of on-going behavior, 


This similarity is well exemplified b 
diagram of adjustive I 


in its essential relatio, 
flict of motives (p. 
ing of a motiva 


ne to begin also to consider 


cir paper is that these 
ation. between problem. solving 
perience of frustration in the sense 
They say: 


ation resulting from a con- 
actors leading to the block- 
ier-circumventing or "solu- 
or learning. Melton (1941). 
alysis of the | earning process and has 
"nn and analysis of learning assumes 


se, both include barr 


follow ing Dashiell, } 
explicitly noted that “ 


- this definitic 


‘These three states obviously corres 
Il) termed the three "original emotional reaction patterns”: 
But whereas the early Behaviorists saw these solely as “reactions,” ncobehavio 
go further and recognize their Motivational and beha 
(see Winsor, 1930). 


pond to what Watson & Morgan (Section 


fear, love, and rage. 


: S cn srties 
vior-determining propertie 
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bie e instances of learning involve in some degree the frustration of a moti- 
ated organism . . ." (p. 669) (Brown & Farber, p. 477)* 

Now as Brown & Farber point out, many learning theorists have 
advanced "nonemotional interpretations of frustration" (p. 477); and, 
in general, the resulting oversimplifications have not been too serious. 
After all, the white rat has been carcfully bred for his docility; and, 
precisely. because of their "viciousness," wild rats have been found to 
be almost worthless as laboratory subjects. But even a highly docilized 
strain of rats still have some spunk and spirit left in them, as the 


experiments to. be cited presently show. With other species this 


dimension of behavior has always been apparent. 

For instance, Finch (1942) has observed that chimpanzees accustomed to 
pressing the handle of a spigot to get water will press with much more than 
usual energy if the water supply is shut off (Brown & Farber, p. 478). 

l'his intensification of on-going behavior (presumably implying in- 
creased motivation) is certainly one of the striking behavioral indices 
of frustration; but in a thoughtful recent paper. Bull & Strongin ( 1956) 
point out that frustration may also result in at least a temporary loss 
of interest in the original goal and "fixation on the obstacle." 

1 foremost with a special type of 
‘hich are related but, for various 
zard a goal, and still operative, 
he obstacle, and aimed at get- 


Thus we are dealing [they say] first anc 
conflict between two orienting attitudes w 
reasons, incompatible—the first one aiming tow 
though impaired; the second one initiated by t 
ting rid of it (p. 531)- 

> their analysis along clinical lines, in 
a highly interesting and suggestive Way but present purposes will be 
best served if we erate to the Brown rber discussion. These writers, 
having abandoned the strictly S—R (frustration-aggression) approach 
of Dollard et al. usefully ‘no anger is admitted as a 
it follows that any 
than aggression arc at least theorctically possible; and they also con- 
jecture that, “Like any unconditioned response, à frustration reaction 
langer] might be conditioned to other (perhaps external) ‘non-frus- 
tration’ stimuli" (p. 499). In conclusion Brown & Farber say: 


It will be apparent from this treatment of frustration as a determinant of 
behavior that no attempt has been made to specify precisely which responses 
The present theory. does lead to the ex- 


Will appear in thwarting situations. 


Vhese writers further develop 


te that, once 
Motivational construct, of several reactions other 


eine B x y 2 i * Pi 
The same general point of view was also adumbrated in Shaffer's 1936 book, 


The Psychology of Adjustment- 
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pectations that the presence of frustration Sil often result in thg Agrees 
cation of responses, that reactions followed by a reduction in frustr ation wi 
be reinforced, and that responses associated a ith frustration in one situation 
will tend to appear in other thwarting situations. But there is nothing in the 
theory itself to imply that aggression, for instance, will occur more f 
quently than withdrawal, or withdrawal more frequently than, sav, prim- 
itivization [regression] (p. 490). 


In an earlier chapter reference has already been made to an attempt 
by Brown & Farber to test the hypothesis that in a situstion originally 
used by Miller to show, purportedly, the role of escape from a fear- 
inducing situation as a secondary reinforcer 


. there might also. (or 
instead) be an clement of frustration and that it might thus be anger 


rather than fear which was the relevant secondary motivator 
forcer. But since this experiment led to negative results, it cannot be 
regarded as a successful test of thc assumption "that reactions followed 
by a reduction in frustration. will bc reinforced.” However, the as- 
sumption “that the presence of frustration will often result in the 
intensification of responses” has provided the inspiration. for a series 
of quite successful experiments by Amsel and collaborators, the first 


of which was reported by Amscl & Roussel, in 
as follows: 


and rein- 


1952, and summarized 


This experiment was desig: 
motivational state. . . , 

Eighteen male albino rats were 
alley into a goal box, then leave 


ned to test the assumption that frustration is a 


trained under hunger drive to run down an 
a that goal box and run down a second alley 
into a second goal box distinctively different from the first one. Their time 
to traverse the alley between Goal box 1 and Goal box 2 was measured dur- 
ing a preliminary period. When this time had reached a stable minimum, 
their running ume and latency in leaving Goal box 1 were measured during 
test trials. In half of these trials they w ere frustrated in Goal box 1 | i.e. the 
expected reward of food was not received]; in the other half they were 
DOES £9. 


An implication OF the: o assumption—that stren 
frustration test trials should be greater than th 


or that on reward test trials—was strong 
[see Fig. 1175]. u 


gth of performance on 
at during the preliminary: trials 
ly supported by the data (p. 368) 


This finding, that a “frustrated” 


2 Tat 1$ more motivated than 4 
satisfied one 


sa > I5 Congruent with conjectures put forward in Chapter 5. 
There the assumption is made that a stimulus with secondary rein- 
forcing properties is one which is capable of alla 
"assurances," arousing hope. Thus it is almost 
tration,” in the objective sense of th 
ment, nonconfirmation of an implied 


ving drive fear, giving 
axiomatic that "frus- 
at term, involves the nonfulfill- 
“promise” and the emergence, 
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at the very least, of the experience previously termed disappointment, 
Which is to say the loss of hope and the return of fear and the resulting 


increase of motivation found by Amsel & Rouss l. This line of reason- 
wing observation by Bull 


ing is independently supported by the follo 
& Stronein: 
term frustration to situations where inter- 


ruption takes on this character of threat and automatically produces the 
action which is found in all “emotional” response 
finitely connote the presence of fear, 
a complete disap- 


We propose + +s to limit the 


mixed type of emergency re 
to danger. Thus all frustration W ould de 
Which is not, however, sufficiently overwhelming to cause 
pearance of the original goal-oriented tendency (p. 532 


2 
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thesis that frustration. may serve 


demonstration of the 
Amsel and Roussel, 1952, p- 367). 


Fig. 11-5. Fxperimental t 
e state of an organism € 


to increase the total driv 


fear and anger are physiologically akin 
activation of the sympathetic division 
(Arnold, 1950; Ax, 1953). And it is 
conceivable that whether al will be subjectively perceived 
ry depend, in part at least, upon certain perceptual 
“sets,” on the part of the organism, i.e, upon 


It has long been known that 
—both being characterized by 
c nervous system 


of the autonomi 
such arous 


as anger or as fea 
and motor factors, OF 
What may be loosely called the 
aad the experience o 
his mutability n 


organisms “intentions.” Presumably 
f having his “fear turn into anger" 


everyone has | ` 
aay be a function of whether attack 


or the reverse; and t 
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or retreat is perceived as the best policy. Could it be bs pins 
some situations, William James was right (sce p- 48) anc n : ^u ped 
afraid because we run (or at least feel that we ought w) D vs > 
are angry because we fight (or feel we ought to)? Here the a pee 
not at all clear, but this much can apparently be said with p "el 
fidence. Just as “appetite” surges up in an organism as it comes a 
to a goal, anger likewise mounts when a barrier suddenly ee a 
Appetite, as conjectured in Chapter 6, has the effect of veni ir g oe 
last additional bit of motivation sometimes needed to ensure success; 
whereas anger, it seems, often gives the organism special energy at 
just those critical points when possible success is about to turn. into 
sure failure (sce Haner & Brown, 1955) 
induced anger does not lead to success, it sc 
will be explored 


. However, if frustration- 
rves another function which 
+ more systematically, in the next section. . 

As will be thus apparent Some progress is now being made in 
articulating the concept of frustration with general behavior theory; 
and the experimental literature is 

literature has just been reviewed by 
reconsidered here." However, 

Opening comments in this conr 


growing apace, Fortunately this 
Amsel (1958a) and need not be 
it is pertinent to note some of Amsel’s 
nection: 

This paper is based on the 
mental behavior must involv 
events—usually the presence 
tion appropriate to some condition of de 
noxious stimulation 


trative events—the 
where it h 


proposition that an adequate theory of instru- 


e three types of goal event: (a) Rewarding 
of stimuli which evoke a consummatorv reac- 
privation; (b) Punishing cuani ™— 
at the termination of a behavior sequence; and (c)Prus- 
absence of or delay of a rewarding event in a situation 
ad been present previously (p. 102). 

It is quickly apparent th 
as capable of becoming conditioned to formerly neutral stimuli 
respectively, bope, fear and anger (although his terminology is slightly 
different). t. A 


at this writer thinks of cach of these events 


as, 


Special attention should nevertl 
demonstration. by 
by frustration, 
at 


heless be called to the fact that, following the 
Amsel & Roussel that 


on-going behavior can be intensified 
Bernstein (1957) has shown thar frustration can 
cast) increase resistance to extinction, 
that anger can motiy ate and 


reinforce new Je 
the Amsel hypothesis (with experiment 


by Seward, Pereboom, Butler, & Jones (1957), 
Collier (1959) have published evidence th 
the manner of fear or hope (see 
tempts to study the rel 


(up to a point 
Adelman & Maatsch (1955) report 
arning, Wagner (1959) has defended 
al evidence) against a criticism advanced 
and Holder, Marx, Holder, & 
"at anger can be 
j also Zbrozyna, 1958). For a review of earlier at- 
ation between frustr 


: ~ Mer 
ation, anger, and learning, see Mille 
(195th, p. 462; also 1957, p. 1278). 


iti ch in 
conditioned, much 1 
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Basically, the position adopted here assumes that all learned, instrumental 
(goal-oriented ) responses depend, to some extent, on classical-conditioned 
implicit responses. These responses are the learned counterparts of responses 
clicited by the three types of goal event—rewarding, punishing, and frustra- 


tive events—outlined earlier (p. 102). 


Revised two-factor theory would differ with this statement only in 


uarded: if the phrase, “to some extent,” were 


holding that it is too g 
able; for it would 


climinated, the starement would be quite unobjection 
ll learned, instrumental (goal-oriented ) re- 
lysis, selected, guided, and controlled by 
1 ave been conditioned 


then assert, simply, that “a 
sponses” are, in the final ana 
the emotions of hope, fear, and anger as they h 
to response-correlated stimuli, independent stimuli, or both (but cf. 
Mowrer, 1960, Chapter 5). 
Especially interesting and rele 
development of the notion that 
termine activating (drive) states, which can be measured as an increase 
in the vigor of ‘behavior which immediately follows the frustrative 


events [see Amscl-Roussel experiment], and are also responsible for 
inhibitory. effects, which are at least. partly responsible for decreases 
in strength of the instrumental behavior which is terminated by the 
frustrative event" (p. 3): Here is a theory of extinction with gen- 
erality and power. The best we have had in the past was the view that 
extinction occurs because habit strength (hope) is offset (through 


counterconditioning) by an inhibitory state created. by the fatigue 
or effort generated bv the nonrewarded repetition of a response. As 
Ammsel indicates, this may still be a factor in extinction; but it is also 
very probable that the “anger generated by nonreward (disappoint- 
Ment) serves to countercondition hope just as docs the fcar which is 
generated by punishment (sce also Chapter j2)." 


vant for present purposes is Amsel’s 
“frustrative non-rewarded events de- 


in a state of flux and change. In his 1958b 
indication of this when he say “The important change 
n in our thinking. about goal factors is in the con- 
nonreinforcement (nonreward). It represents a move- 
atment of nonreinforcement (following later Thorn- 
yehavior (cf. Amsel, 1951, 19583) and a return to 
tin Spence's treatment. of discrimination learning 
in which nonrew ard is regarded as an active 
es. In his 1936 paper. after referring to the 
"The second principle to be used is 
hat when a rcaction is not rewarded. 
ic, when the final consummate prevented from taking place, the 
are weakened ..- and it is assumed that this weak- 
(p. 430)" (pp. 3-4) In this paper Amsel 


*Theory in this area is obviously 
article, Amsel gives some 
Which seems to be going o 
ceptualization of the role of 
ment away from Hull's tre 
dike) as a nonactive factor in | 
an earlier position, most explici 
in the "3oies. (Spence. 1936, 1937). ' | 
hibitorv tendenci 
Spence W rote: 
which states t 
ry response is 


factor determining inl 
principle of reinforcement, 
that of inhibition or frustration, 


excitatory tendencies -+ - 


ening is due to an active, negative proces 
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How much progress has been made during the past 30 years in our 
understanding of the whole problem of "unlearning" can be nicely 
illustrated by reference to an article published by Winsor in 1929. 
At that time, as Winsor pointed out, William James’ old Law of 
Exercise (1890, Vol. I, Chapter 4) was still widely ac epted; and he 
cited a later formulation thereof by Gates (1928) as follows: “When- 
ever a modifiable connection between a situation and a response is 
exercised, other things being equal, the strength of that connection is 
increased” (p. 282). Having become familiar with the phenomenon of 
experimental extinction (as reported by Pavlov, 1927) and with Dun- 
lap's principle of negative practice ( 1928), Winsor cogently remarked: 


An examination of almost any psychology text which describes the learn- 
ing process will show that this dual aspect of adaptation has not been taken 
into consideration. It appears from the texts that the 
repetitive stimulation as inevitably increa ing the probability that the re- 
sponse will continue to be positive. It seems never to have occurred to them 
that exercise might function in developing à negative response. In fact they 
try to account for adaptation without this type of response and when they 


are confronted with the inhibitive behavior assume that it is merely a limita- 
tion of the law of exercise (p. 395). 


authors conceive of 


And then, in reference to the results of an experiment which Winsor 
himself reported, he said: 

If we may assume that the increased secretion of the parotid [salivary 
gland] to the sight of food represents a typical organic habit and that under 
the conditions of the experiment it was "broken? or eliminated, these data 
become very significant to students of behavior. They afford a study of the 
factors which cause a positive or excitatory reaction to be 
negative or inhibitory reaction. The factor of exer 
If any one aspect of the total Situation or organic influence could be singled 
out as of especial potency in the fixation of inhibit 


te: 1 ion it would be repetition. 
Other factors remaining constant, this factor of exercise is scen to facilitate 
does not exclude the factor of fatig 
indicated, he is inclined to give gre: 
reward. While granting that repe: 


changed into a 
ise is rather clearly shown. 


ue or effort in e 
ater weight to the 
ated nonrewarde 
cause an organism to get tired (sce Section III), 
cause the organism to get mrad, 


condition the positive emotion 
to the sensory feedback. from th 
Spence has abandoned his o 
followed Amsel in reve 
“Spence (1956, p. 151) has forsaken h 
simple experimental extinction, 
conditioning hypothesis” 
in general —and 


xtinction; but, as already 
emotional effects of non- 
d performance of an act may 
i it may, more importantly, also 
And this negative emotion will then counter- 
chope) which has previously been conditioned 
n the act in question, Interestingly enough, although 
riginally in in these "matters, he has not 
Say Noble & Alcock (1Q58): 
abit-strength interpretations in favor of 
thereby adopting a Giuthrian type of counter- 


ume (Pp. 410. For a recent review of the 
a nibliography Of 257 titles, sce | 


J Hullian positic 
rung to his earlier. views, 


frustration concept 
aawson & Marx (i958), 
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An important factor in the breaking of this 
In order to conserve energy nature 
as to utilize adequate signals. Both 
processes appear to be subject to the same important basic laws. Whether the 
reaction develops into a positive or negative habit depends upon the biologi- 
cal significance of the situation. As the significance of the signal to the 
organism changes, persistence of that signal causes a change in the response 


(pp. 3947395)- 


inhibition as well as excitation. 
organic secretory habit, therefore, is use. 
is equipped to eliminate false signals as w ell 


Obviously, as we now know, the sheer occurrence of a response is 
no guarantee at all that the likelihood of its recurrence will be in- 
, Vinsor correctly points out, the repeated per- 
lead to its inhibition rather than facilita- 
imp the Law of Exercise, it can be 
sume, is in the nature 


creased; in fact, as V 
formance of a response may 
tion, If, however, we somewhat r 
All learning, we now 
sine qua non for conditioning is the conjoint 
led conditioned and unconditioned stimulus. 
At this level of analysis, so far as we know, the “Law of Exercise,” 
or repetition, is unexceptionally truc: Every time a CS is followed by 
the UnCS, the likelihood that the CS will produce the R originally 
clicited by the UnCs will be increased, provided that we are careful 
isively tO emotional responses and do not extend 
it to overt behavior as well (see Section VD. And when acs is not 
followed by the usual UnCS or when it is followed by a different. op- 
posite type of UnCS, the original conditioned d ous is extinguished 
or (in the case of counterconditioning) replaced by an opposite type 
of reaction, It is only when we TY to apply these principles to be- 
havioral, rather than pur . autonomic reactions that diffi- 


culty and confusion artise. 


“Thus a remarkable evolution of thor i e 
curred, For a time it looked as if the Law of Exercise was to be com- 


pletely eclipsed bv the Law of Effect; but as the latter has been 
probed and reformulated in the course of preceding chapters, we find 
al approximation of facts which are more precisely 
sical conditioning—and here “exercise,” in 


given a new validity. 
of conditioning; and the 
presentation of a so-cal 


to apply this rule exclu 


clv cmotional 


ight may be said to have oc- 


that it is only a glob 
explained in terms of classi 
the sense of stimulus repetition 


“in contiguity,” is of the essence. 


V. Maier's Frustration-Fixation Hypothesis 


reviewed in the preceding section adds up to a strong 


The evidence ne p E t 1 
ation is often an important factor in the abo- 


presumption that frustr 5 SU » e s x 
lition of “habits.” It was therefore surprising that N. R. F. Maier, in 


1949, should advance the notion that frustration, under certain special 
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circumstances at least, results in their fixation, rather than elimination. 
His argument went as follows: 


If [in a Lashley discrimination appartus, see Chapter 12] the ca rds are 
latched in no regular order (i.c., neither a particular card nor a particular 
position is consistently reward or punished), then there is no response that 
will permit escape from punishment, In such case the animal normally shows 
a stage of variability in its choices and soon thereafter it refuses to jump. 
This resistance to jumping may be overcome by giving the animal an ele tric 
shock at the jumping stand, prodding with a stick, or blowing a blast of 
air on it. Under these conditions the animal can be forced to jump. We 
speak of this situation as the insoluble or no-solution problem and regard it as 
frustrating both because it is a problem that cannot be 
pressure is applied to the animal to force a response. 

After a short while in the insoluble problem situation and with pressure 
applied to force behavior, the animal develops a response to the situation that 
has no adaptive value in the sense that it is adequate to the situation or in 
the sense that it is superior to any number of other possible responses. Never- 
theless, the appearance of the behavior is associated with a decline in resist- 
ance to jumping. Thus an animal that is forced to respond in the insoluble 
problem situation may always choose the card on its right, despite the fact 
that this choice is punished on half the trials, This type of response is not 
selected by the method with which reward and punishment are used. At the 
same time it is not a mere random response but is consistently expressed and 
so must be considered as a response to the situation (pp. 26-27). 


solved and because 


Equally striking, of course, were Thorndike’s sudden repudiation of 
the negative half of his celebrated Law of Effect (see Chapter 2) and 
Spence's suggestions that there was probably no primary gradient of 
reinforcement (Chapter 10). While useful in stimulating new 
and clarifving theory, neither of these observations has, however, 
proved basically sound; and it now appears that much the same verdict 
is in order as regards Maier's novel thesis, Hull once remarked that 
the “ingenuity” with which psychological experiments are designed 
is sometimes such that the experimenter himself is misled, no less than 
his subjects. At the very least, we may say that the M 
involves a great many variables and that the frus 
hypothesis, as advanced by Maier, 

In a review of Maier's book, 
suggested that the picture 
following lines. Originally, 
the Maier-ty pe exper 1 


rescarch 


aier procedure 
tration-fixation 
seems to neglect some of them. 
l published in 1950, the present writer 
might be clarified by thinking along the 
during the discrimination-tr ii 


iment, the rats are motiv 
warded, for jumping tow 


they are punished, by 
for ; 


aining phase of 
ated by hunger and re- 
ard the correct stimulus card, by Food, Also, 
a bump on the nose and 
jumping toward the incorrect stimulus ¢ 


a fall into a net below, 
ard. By carefully attend- 
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ing to business, the rats can, of course, bring the incidence of reward 
up to roo'« and reduce the incidence of punishment to oo. And 


this is precisely what they tend to do (see the Chapter-12 discussion 
). But when the “rules” are changed and 


of discrimination learning 
andomly locked or loose (instead of the “correct” one 


the cards are 7 
“incorrect” one always locked, as during 


always being loose and the 
the preceding training). the 
“frustrated.” Now the best they can proba 
(rewarding) and zo“ incorrect (punishing); and it is not surprising 
: refuse to “play.” and have to be forced to jump 
under threat of punishment of some kind, from “behind.” Neither is 
that, under such coercion, the animals, although con- 
tinuing to jump. yandon the attempt to “follow” the (now meaning- 
less) positive cuc card from one side of the apparatus to the other on 
Sheweasive eris and rever (“regress”) to the simpler, casier, and vo 

jumpir 


less rewarding practice of always ji 
what Maier called a “fixation.” 


there is nothing in the least 


animals are—understandably enough— 
bly do is 50% jumps correct 


that they “get mad," 


it remarkable 


rg to the same side, i.e, de- 


veloping a “position habit’ —or 

Viewed from the proper perspective, ) 
paradoxical or exceptional about this finding: the animals have learned 
a discrimination habit, conditions are changed so that this habit can no 
longer operate effectively, and the discrimination habit is soon aban- 
doned! Frustration can be said to result in “fixation” only if attention 
is shifted to a different mode of adjustment, which is sizzpler than the 
discrimination. habit and which persists only because of the coercive 
Maier has been inclined to refer to such a 
or without a goal.” Eglash (1951) is probably 
en he suggests that there is a change in goals: 
est in obtaining food (which the disc imination 
ectly. provided 100*« of the time) to the Jater 
s on the jumping stand (which thc 
as just as well as the 


threat. of. punishment. 
position habit as “beha 
more nearly correct wh 


from the original inter 
habit, if practiced perf 
interest in avoiding p 
position habit did 100%% 
esumably, 
ager function 


unishment 
of the time—which w 
more difficult “discrimination” habit, 


al in the way it originally was, 


more complex and, pr 
Which was now no lot 
could do). 

Maier has held that when 


exceeded, there is a categorical loss of I . 
resulting "abnormal" stereotyping of action. Our alternative suggestion 
g“ al” s y ping 


is that the reversion to a position habit, under the circumstances de- 
scribed, is realistically justified and does not at all imply a loss of 
normal flexibility and intelligence. This view is supported by the fact 
that some of Maier's animals do vot “fixate” but continue to experi- 


an organism's “frustration tolerance" is 
A Hd -— 
f behavioral adaptivity and a 
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ment with alternative right and left jumping. Moreover, of those 
animals which do develop the position habit, Eglash (1954) has made 
the following pertinent observation: 


Whereas fixation suggests rigidity or perseve 
mains as fle 


ion, the fixated animal re- 
le as the normal. This flexibility sometimes [takes] the form 
of a surprising ingenuity. . . . In addition to displaying a variety of re- 
sponses, as described by Maier, the fixated animal may drive directly into 
the net, or leap so lightly that it fails to reach the window. It may assume à 
passive role: the air blast designed to force the jump blows the animal off the 
stand (p. 241). : t ; 

This adaptability suggests substitute, rather than “fixated,” behavior (p. 
242) [see also Ellen, 1956] 


Much has been made of the fact that, once an animal in the Maier- 
tvpe experiment adopts a position habit (ic. starts jumping cgn- 
sistently to either the right or the left side of the discrimination 
apparatus), its resistance to leaving the jumping stand diminishes. 
And since such a pattern of behavior involves abortive (punishing) 
jumps on 50% of the trials, it has been argued that the normal re- 


sponsiveness of the animal to punishment and reward h 


as been lost. It 
should be remembered, 


however, that this incidence of failure (and 
punishment) is vo worse than would probably be 
animal continued to show some pattern of 
besides, the position habit (as already 


experienced if the 
alternating response; ^ and, 


noted) is casier, simpler. 
But over and bevond these consider 


interesting possibilities that should be 
Maier's book pre 


ations there are two other 
briefly noted. In the review of 
iously cited (Mow rer, 1950C) 
once the position habit gets well under 
to gain momentum through the mech 
by Whiteis (1955, 


it was suggested that, 
Way, it may have a tendency 
anism involved in the experiment 
1956) which has already been 


alluded to in Chapter 
6 (and will again be considered in Chapter 


12). Here fear which is 

"The word "probably" occurs in 
will be recalled that in the frustr 
cards is still loose and the 


this sentence for the following reason. It 
ating part of Maier's experiment, one of the 
i other locked on any given trial, just as in the discrimina- 
tion training, except that now the cards are locked or loose without respect to 
whether they carry the positive or the negative stimulus cue. Therefore, it would 
still be conceivable, though most improbable, that. an 
jump to both sides of the apparatus, would just h 
on successive trials, was unlocked, 
ishment would be experienced. B 
course, astronomically 
locked card. On the 
of the time and 


animal, in continuing to 
appen to select the side which, 


Thus, 100% food reinforcement and no pun- 
ur the odds a 


high—as are the odds 
average, animals wou 


a loose one equally often, 
sistently to one side of the appar 


gainst such a performance are, of 
against a rats always choosing the 
ild tend to choose a locked card 50% 
regardless of whether they jumped con- 
atus or continued to jump to both sides, 
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reinforced by punishment at one point in space generalizes “back” to 
roduces what, ostensibly, is a sort of 
Thus, in the Maier experi- 
ch is intermittently 


another point in space and p 
behavioral equivalent of perpetual motion. 


ment, it is conceivable that the punishment whi 
he position habit would produce 


experienced by the rat in performing t 
anding stand and tend to 


fear that would generalize to the jumping st 
* considerably longer than might other- 


keep the jumping habit “going 
Wise occur (cf. Lichtenstein, 1957)- 
Furthermore, reference has just been made to the fact that, in the 
“fixated” position habit, punishment is experienced “intermittently — 
50'( of the time the card is locked gets à nose bump and 
a fall, but so* of the time the card is loose and the rat's jump carries 
À re there is food. And we have already 
seen (Chapter 4) that intermittent reinforcement, in gencral, tends to 
ance to extinction" than does continuous rein- 
“fixation” in Maier's experiment 


and the 


him into a compartment whe 


produce greater "resist 
forcement, so that it might be that the 1 
arises, in. part at least, from this phenomenon (see also Chapter 12). 
apparent that Maier’ contention that frustration 
sometimes fixates a habit rather than disrupting 1t !5 by no means 
logically. coercive. Instead it seems that we are justified in continuing 
to assume that the absence of reward, where reward is expected but 
nok reviewed (aris expected in a given amount but less is received, sec 
and Leary, 1957). generates an affective reaction 
This emotion has now been shown to bc 
and, on occasion, to result 


In any event, it is 


‘Vinklepaugh, 1932. 
commonly known as anger. 

capable of intensifying action in progres 
i ' attack on a barrie 
tensified effort nor attack succeeds, 


in aggression toward or r interposed between organ- 
ism and goal, But when ncither in 

frustration and the resulting anger have a different effect. They result 
ifia lesening of "habit strength," or hope, by virtue of the fact that 
anger (like fear in the more drastic punishment situation) produces 
1 ioning (Section IV). asa result of which hope is gradually 


estroved. In the Maier experiment, hope 
sitive cue card in the training procedure; 
periment, the positive card became no 


countercondit 
nullified and habit strength d 


had been conditioned to the pt 
r part of the ex ! 
gative one. In fact, we may say with equal 
punishing. so that the rat abandoned 


but then, in the late 
ie than the nc 
it it became equally 
“following” 
bly) toa simple position habit (always 


more rewardit 
justification that i 
the earlier habit of 
(fatalistically but not unreasona 
right or alwavs left). By the procedure employed in the second part 
of the Maier experiment, discrimination was indeed rendered insoluable; 
and the subjects might w ell be said to have been “frustrated.” But also 


the positive cuc card and reverted 
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we must note that in the no-solution period what was really happening 
was that the differential reinforcement for discriminating was totally 
eliminated (changed from 100/0 to 50/50); and since discrimination was 
harder work (and now no more rewarding) than was fixation, by the 
principle of least effort (Sumner & Keller, 1927, Waters, 19375 sce also 
Section III), one would expect exactly the result obtained. In fact, 
such an outcome would follow from principles discussed much earlier 


in this book (Chapters 2 and 3) and does not require that the concept of 
“frustration” be introduced at all. 


VI. The Differential Effect of Reward and Punishment 
on Overt Behavior and on the Emotions 


A number of writers in psychology have attempted to handle emo- 
tions "just like any other response," "The reader may have been 
surprised to find (in Section II) that Watson 
of Bchaviorism, 


. as the chief exponent 
even recognized the existence of emotions. The fact 
is that, in the sense in which we are us 
For Watson, an emotion was merely 
behavior; and just as he assumed th 
be conditioned, so also could 


ing that term, he did not. 
another example or exhibit of overt 
at other observable reactions could 
an emotion be. For Watson, emotions, in 
the sense of a subjective state (or “intervening variable’) with moti- 
vational properties, simply did not exist (at least not in his formal 
theoretical system). His position was therefore 


one but also, for numerous reasons already: 
unsatisfactory, 


a thoroughly consistent 
discussed, thoroughly 


Hull, Miller, and other “re 


inforcement theorists" have similarly held 
that the emotions 


are subject to the same principles as govern the 
learning of responses in general. Since they assumed that drive re- 


duction, or reward, was the sole reinforcement mechanism for the 
development of “habits,” they 


that they acknowledged them) 
reward. This position leads to 
notable of which is the 
(drive reduction), 
already noted, 


assumed that emotions (to the extent 
were also "learned" on the basis of 
a number of manifest incongruities, most 
notion that fear is strengthened by reward 
rather than by punishment (drive induction). As 


í this is one of the basie issues on which the present 
approach differs from that of Hull. Here we are assuming that be- 


havioral responses are categorically different from emotion 
the former are “voluntary” 


and punishment (and not C 
the latter are involuntary 


al responses: 
and subject to influence through reward 
onditionable, strictly speaking), whereas 


and conditionable and not subject to control 
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through reward and punishment, or at least not in the same way as are 


the overt-behavioral responses.” 
Emotional responses, it is assumed 
counterconditioned; but because they 
principle from that involved in behavior: 
spond to reward and punishment as such. For examp 
an indicator of hope that food will soon be forthcoming) cannot, so 
far as we know. be strengthened by rew ard, at least not in the same 
" ^or some other "instrumental" act 


(Sections HI and IV), can be 
work on an entirely. different 
al responses, they do not re- 
lc, salivation (as 


Way that bar pressing, paw lifting, or 
can be. If we should want to teach a rat to press a bar barder and 
harder, as a means of obtaining food, all we would have to do is to 
withhold reward for relatively weak responses and give it only for 
Increasingly vigorous ones. Under such a regime the incidence of 
"hard" responses would increase. 

Now consider, by contrast, how differently. the salivary response 
behaves, So far as is known to this writer, no attempt has ever been 
made to perform an experiment in which this response would be 
treated like an instrumental act, would be required to occur 1m 
specified magnitude before "reward" would be forthcoming. If such 
1 ssful, a Pavlovian dog could be turned into a 
quiring the dog to secrete a 


ies 


an experiment were succe 
vertiable “saliva factory, 
little more each time the CS was pres 
presentation of the UnCS (food). If t 
widely known, And when the Polish neurophysiologist, J. Konorski, 
Was recently in this country. he said he would regard such an ex- 
: n worth trying, 50 certain was he of the outcome: 


namely, that it would not be successful. Salivation, like conditionable 
à al, is “reinforced” by the paring of the CS and UnCS 


and each omission of the UnCS would presumably weaken the salivary 
reaction to the CS. not strengthen it (but see Perkins, 1955; also 
Chapter 12). And Konorski went on to say that, quite apart from 

] work previously cited on the emotions of 
he reaction of nausea can be con- 


ds simply rc 
` ented, as the precondition for the 
his has ever been done, it is not 


periment as not eve 


responses in gener 


menta 


D B E 
In addition to the exper! 
ld be noted that t 


fear, hope, and anger, it shou 


ditioned and probably then provides at least : c com 
monly termed lisgust. Loucks (1935) and Konorski (1950) both cite studies in 


which the sight of a hypodermic needle that had been used to inject. morphine 
into dogs became capable. alone, of producing definite signs of nausca; and per- 
sons who have been sev ercly seasick or airsick report that the “mere sight" of a 
ship or plane can reinstate the experience in atrenuated form. Thus far, then, we 
May say that onc "positiv c^ and three "negative" emotions have been identified 


and operationally defined: hope (or love) on the one hand and fear, anger, and 
disgust on the other. 


a species of the experience. com- 
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the difference in the nervous mechanisms controlling emotional and 
behavioral responses, there is the fact that the former have no kin- 
esthetic (or related forms of) feedback; in fact, the nervous system 
which mediates the occurrence of the emotions (ie, the autonomic) 
is entirely a motor (efferent) s "stem; and the knowledge of the occur- 
rence of such responses is derived from sense organisms that are con- 
nected with the central nervous system. Moreover, there are many 
autonomic responses (e.g., the pupillary response) which can occur, 
and even be conditioned, without the subject’s even “kt 


about 
it (Gerall, Sampson, & Boslov, 1957). 


NOW ING 
But perhaps the most striking illustration and indication of the 


way in which emotions and motions differ is provided in the realm 


part of the body which 
) muscles can, as everyone knows, 
| if the movement occurs and produces 
certain characteristic response-correlated stimuli, these stimuli, if fol- 
lowed by drive increment, acquire the capacity to arouse fear and 
thus inhibit, or at least modify, the response on future occasions. But 
let us consider a fear response which is ¢ 


danger signal: 


of fear. A motion, or movement, of some 
is controlled by skeletal (or "striped" 
be inhibited by punishment; i.c. 


alled out by some objective 
if this response were “punished,” the effect would be to 
augment the response, not inhibit it, since danger signal followed by 
drive increment would be the very condition that brought the fear-to- 
signal response into existence in the first place. d 

And just as one cannot weaken a fe 
neither can one stren 
previously seen, 

Finally, 


ar reaction by “punishing” it. 
gthen it through reward: reward, as we have 
acts to countercondition fear, not reinforce it. 

we need to make explicit one otl 
already implied in what has gone before. 
ploring the possibility: that w hen hope 
extinction), the subject not only does not get positively reinforced (by 
food or the like) but, in addition, experiences the distinctly negative 
emotion of anger, which tends to countercondition the hope reaction. 
Now we must ask, what happens when a fear is not confirmed? If a 
stimulus (or stimulus compound) which has previously produced fear 
is recurrently associated with primary drive reduction rather than with 
drive induction, the fear is, of course, gradually nullified and ultimately 
replaced, or Counterconditioned, by hope. But what about the situa- 


tion where, rather than being thus counterconditioned, the fear is sim- 
ply not confirmed, not reinforced? Here it might seem that the prin- 
ciple of unlcarning through counterconditioning Meets a limitation. 
But not so. When a danger signal terminates without the customary 


ter consideration which is 
In this Chapter we are Cx- 
15 not confirmed (as in so-called 
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c organism, as WC well know, 
ondary decremental reinforce- 
of counterconditioning fear 


reinforcement (noxious stimulation), th 
experiences relief, which is a form of sec 
ment and ought, therefore, to be capable 
itself, Thus, just as nonconfirmation of hope produces anger (and/or 
fear) which counterconditions hope, so also does nonconfirmation of 
k which likewise can function as a counterconditioner. 
There is, however, à limiting factor: since frustration (and anger) can 
v so long as there is hope. and since relief can be 
experienced only so long as there is fear, the counterconditioning, in 

d c soint of neutrality, no reaction, 


roceed only to the [ 

if, in the first case. reduction of a drive other 
the counterconditioning can, obviously, be 
the opposite type of emotional 


fear produce relief, 
presumably occur onl 


both instances, will p 
“extinction,” Wher 
than fear is introduced, 
null point, over to 


carried beyond the 
all learn- 


reaction. 

‘Thus we arrive at the conclusion that, in the final analvsi 
ing (at least of the emotions; cf. Mowrer. 1960, Chapter 5) involves 
conditioning principles and that all systematic behavior change is de- 
rivable. therefrom. So-called instrumental 
(Chapter 2) work on ry responses and are derivative 
phenomena; the basic operation is cl: 
by the same token, all unlearning 
counterconditioning. and not simpl 
the conditions of reinforcement whic 
view of the matter follows logically f 
tion and possesses broad explanatory P 


conditioning procedures 
ly with volunta 
: i sical emotional conditioning. And, 
of emotional reactions involves 
y the withdrawal or absence of 
ch have previously prevailed. This 
rom our general systematic posi- 


otencvy- 


VII. Some Experimental Approaches 


to the Problem 
ission of the way in which emotions and 
and punishment has been largely 
ist ask What is the evidence? 
is that the evidence is, as Vet. fragmentary 
first, and still one of the few, experimental 
c on this problem was reported by Gwinn 


his study on this premise: 


discu 


Up to this point our 
I 
reward 


motions are affected. by 

“eheuretical.” We now Tm 
The first thing to be noted 
and sparse. Apparently the 
attacks which have been mad 
in 1949. This investigator began 


specifically: 


shing stimulus is compatible with the punished 
atber than inbibit an act motivated by fear. 


to the puni 
ment will strengthen the fear-drive motivat- 


„jll facilitate T 
{by the punish 
( 260). 


If the response 
act, punishment 
i he fear produced 
ing the punished act (p- 
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(ous 
z] 
s| 
5 
Cue 
4 Fig. 11-6. Floor plan of appa- 
ratus used by Gwinn (1949. 
P- 261) in a study of the cf- 
fects of punishing a fear-moti- 
3 vated escape response in rats. 
2 


Starting hopper—~ ae cage 


The apparatus used by Gwinn is sl 
training, a rat was dropped into this 
the entire grill floor clectrically charged, except for section 8, which 
was the floor of the “escape cage.” After some initial confusion, the 
subjects all learned to run quickly from the “starting hopper” to the 
“escape cage.” Then the subjects were put into the starting hopper 
with no shock on the grill and only fear to motivate them. Again they 
ran around the alley and into the escape cage. But now a complication 
was added: shock was reintroduced in sections 6 and 7. so that it acted, 
in effect, as a “punishment” to the animal for running the maze. ‘The 
result was that the animals nevertheless contin r 
fact ran it more energetically for a time th 
although eventually they were inhibited and 
entry to section 6. Of this finding, Gwinn s; 


hown in Fig. 11-6, In preliminary 
i 8 F Ji 
circular maze at section 1, with 


ued to run the maze, in 
an did control animals, 
“froze” in front of the 
aid: 

The punishment facilitated the punished act, 


with the intensity of the punishment. There w 
until after many repetitions of the punishment 


and the facilitation incr 
‘as no evidenc 
(p. 269). 


Thus the results are somewhat ambiguous: 
the first effect of punishment on a fear-motivated act is to facilitate it. 
the act is finally inhibited. Although interesting, it is obvious that this 
experiment does not bear precisely upon our problem, which has to do, 


they show that although 
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s, but the fear itself. Our assumption would 
was to produce in his subject a 
and the desire to get 


not with fear-motivated act 
be that what Gwinn did, in substance, 
conflict, between (a) fear of the maze as à whole 
and (b) a more specific fear of entering into 
cd as a sort of “bottle-neck” for the escape 
response. At first the shock experienced in sections 6 and 7 apparently 
acted, through generalization (Chapter 12), to increase fear of the 
alley as a w hole and to accelerate escape therefrom. But eventually a 
discrimination set in (as Gwinn himself, p. 268. observes) and the 
subjects stopped running, i.c., fear of entering sections 6 and 7 became 
greater than was the rats’ fear of the maze as a whole. In other words, 
the rats learned that what happened in sections 6 and 7 was different 
from what happened (702. as arlier) in sections 1 to 5. 
(See also a somewhat similar st Kamin, & Wynne, 


1953 
_ In this experiment, therefore, th 
for being fraid, but for a particul 
fear. Here, as noted, there was the 
and eventually to become 50 great as tO ! 
motivated action. The question, What happens if fear itself, rather 
than a fear-motivated act, is “punished”? thus remained unanswered. 
A more direct attack upon this problem has recently been proposed 
by Perkins & Hertzler (1958). Heart rate, as 2 direct physiological 
index of fear, will be uscd; and, after presentation of a signal, a strong 
be pr sented as a function of whether the heart 
rate increases or decreases. "Two groups of subjects will be run. For 
Group A the intensity of thc shock will be a positive function of the 


into the "escape cagc" 
sections 6 and 7, which act 


opposed to € 
udy by Solomon, 


c subjects were actually not punished 
ar overt response they made to their 
possibility for conflict to develop 
inhibit the original fear- 


ora weak shock will 


f shock." i.e., the more the heart rate goes 
up, the stronger will be the shock; and “the shock intensity will be a 
negative function of heart rate for Group B." i.c., the greater the in- 
Crease in hcart rate, the Jess will be the shock. Here, in kek ss 
increased heart rate (fear) will be punished, whereas in Group B it 
will, in effect, be rewarded. To the extent that heart rate 1s 2 valid 
index of fear, revised 97 factor theory would predict that punish- 
ment? will augment the “response” (heart acceleration) W hich it fol- 


lons ani chat sree will have the reverse effect. This, of course, 
ch other (n onses are affected 


heart rate just before onset © 


is not the way in. whi onemotional) resp 
by punishment and reward. : j UR on 

Coppock t 1955) has published an article with the intriguing title, “Re- 
sponses of | Human | Subjects to Own Galvanic Skin Responses. 
Since the GSR is commonly tak an index of fear (or "anxiety" ). 


Their 
en as 
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it might be expected that the question of what happens to a fear when 
rewarded or punished was here under investigation; but such s as nak, 
or at least not directly, the case. However, the paper contains this 
interesting sentence: 


An unpublished observation made in collaboration with Dr. Robert Glase 
in 1949 suggested that the amplitude of GSR's to standard stimulation varies 
in some subjects with their stated intention or set to “feel fear or to "feel no 
fear" (p. 25). 


Whether this observation could bi 


€ confirmed by more systematic 
experimentation is uncertain; 


but it is at least suggestive of the phe- 
nomenon of "repression." In clinical liter 
read that a given individual * 
the inference might follow tl 
through a traumatic 


ature it is not uncommon to 
‘repressed his fear" (of this or that); and 
yat such an inhibition of fear was achieved 
( punishing) experience. of some kind. Despite 
many conjectures as to how (and if) such repression actually occurs, 
it is still not well understood; but this passing reference to repression 
provides a natural transition to the next topic. 


VIII. The Problem of Conflict Re-analyzed 

Reviewing the discussion up to this point, we may say that, within 
the framework of revised two-factor theory, learning and "unlearn- 
ing" are conceived as follows. From a point of neutralitv, learning can 
move in either stimuli (extrinsic or response-pro- 
duced) may acquire the Capacity to elicit cither hope or fear (or 
certain other “aversive” emotions) ; and both types of learning can 
be counteracted by the Opposite form of reinforcement Or by non- 
reinforcement (which, in the final analysis, may ( 
conditioning; see Section VI), Bur there is als 
noted: whereas nonreinfor 
to the point of Neutrality, 
carry learning bevond ne 


of two directions: 


also involve counter- 
also this difference to be 
arily reverse learning only 
reinforcement w ill, of course, 
to the Opposite form of con- 


cement will ordin 
contradictory 
utrality, over 
ditioning. With these principles one can 


; 
the known facts of res 


go a long way toward deriving 
ponse learning (and unlearning ) and place 
learning. (and unlcarning) (see Chapter 9), 5 

Jut. what, then, of conflict? Is not the 


very possibility. of its occur- 
rence thus excluded, at least. in the ; 


Ory, quite as completely as in 
Law of Effect? The earlier ver- 
as an advance over the Law of Effect in 
that it took conflict into account and could account for it; but it was 


flict resolution; Now the 


Thorndike's early formulation of the 
sion of two-factor theory w 


weak in that it could not readily: explain cor 
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its other advantages, secms 
conflict than did Thorn- 


revised version of the theory, whatever 
in danger of having no more place in it for 
If. on the basis of past experience, a given situation or 


dike's theory. 
positive significance (hope, promise, 


response has acquired, let us say, 
and if that situation or response is NOW asi 
will not the meaning of the 
ally and progressively, without 


secondary reinforcement), 
sociated with negative reinforcement, 
7 automatic 
If the sign function, or sign-ificance, 
through either so-called. ex- 
it is not casy to see 


relevant stimuli be modified, 
conscious conflict or antagonism? 
directly changed, 
ounterreinforcement, 
and the behavior vacillation 
t could come about. One might expect 
rather that, with withdrawal or reversal of reinforcement, attitudes, 
expectations, and action of one type would quietly give way to 
different. beliefs and behaviors. ‘This, however, is not what typically 
occurs, Conflict intervenes as 3 real and often obvious phenomenon. 
explained, it at all, within the framework 
g theory? 

stration will be 
the end of a sm 
av from the stick (see also Deese, 
ao was conflict. How to explain 
he stick and food have, 
taken on the 


of a stimulus can be 
tinction or more explicit e 
how the inner tension and restlessness 


which characterized. conflic 


How can its occurrence be 
of revised two-factor learnin 

From Chapter 2 a demonstr 
shocked for cating food from 
nately move toward and then aw 
1952. pp. izi-122). Here, manifestly, 
it? The stimuli associated with approach to t 


as a result of their association with electric shock, l 
fear. On the other hand, stimuli associated with 


n thus associated and conditioned. 
mounting fear as a result of 
it reverses its line of pro- 
h the latter behavior take 


recalled in which a rat, 
all stick, would alter- 


Capacity to arouse 
retreat from the stick has never bee 
animal experiences 


Consequently, the ) 
i f whenever 


approaching the stick and 
gression. Hence, the stimuli ! 
ment pot acking-away behavior de- 
` Bur while such behavior reduces the drive of fear, 
so when the fear of shock is reduced, the 


and will, sooner or later, instigate 


relic 
associated wit 


opo acy and b: 
on secondary reinforce ences an 


velops as a “habit. 
it does not lessen hunger; 
hunger and hunger fear 
the rat once again to move 

The kev to the conflict. obse 


remain 
forward." = 

rved here is the fact that different 
ce in his own behavior. In his 


observed a similar instan 
flowers that needed watering. 


ES 

The author recently SC nets 

back yard one summer g HE ug tone 

This stimulus put him in motion tow ard the watering can. But the can was at 
some distance and the hydrant still more * 
anticipated weight of the water on the return trip. , 
ment toward ihe warcring Can stopped, then a glance at the flowers reinstigated 
are d E 5 " zi 


It, etc. 


evenin 
remote, There was, moreover, the 
The evening was hot. Move- 
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emotions are associated with different stimuli: fear with stimuli corre- 
lated with the forward motion, relief with stimuli correlated with back- 
ward motion. However, there is a further complexity. Moving for- 
ward is associated not only with fear of shock but also with hope of 
food; and moving back is associated not only with relief. from shock 
fear but also with awareness of the persistent hunger. Hence, the very 
same stimuli tend, in both instances, to have contradictory implica- 


tions; and we may well imagine that their meanings will, as a. result. 


tend to "even out." By this is meant. that the opposing meanings, of 
hope and fear, will undergo a sort of 


algebraic summation which will 
eventually bring their 


net effectiveness. somewhere near zero, OT 
neutrality. If the various primary and secondary drives thus involved 
are well balanced, we might expect, as a final result, that the subject 
would abandon its oscillatory behavior and come to rest. |f. this 
equilibrium of forces is disturbed, through mounting hunger or de- 
clining fear or both, then the conflict may “break out" again, and 
no enduring peace will be found until cither (a) another way of ob- 
taining food is discovered, (b) the animal discovers that the shock 
no longer occurs as a result of eating from the stick, or (c) the ani- 
mal grows weak, loses its hunger, and dies. We, therefore, seem to 
have here at least à general framework for 
the major facts of conflict and one which is entirely congruent with 
the premises of revised two-factor theory," $ i 

In an earlier Section of this ch 
there is an implied 
noted. 


accommodating many of 


pter, on extinction and frustration, 
inconsistency Which should now be explicitly 
There it is posited, first, that the anger which results from 
failure of an expected satisfaction often serves to intensify activity 
in Progress and to increase (at least temporarily) resistance to ex- 
tinction; at the same time it is also assumed (by some of the writers 
cited) that anger is the negative or ‘ c 

crated in so-called extinction. situations and which therefore serves 
to countercondition (or counteract) the hope involved in a given 
"habitual" mode of response. Grossly, x 
tion here: anger is assumed both to re 
cline of a habitual res 
reward is withdrawn, 


“aversive” emotion which is gen- 


at least, there is a contradic- 
duce and to produce the de- 
ponse which typically occurs when accustomed 


"Miller (1944), although operating in a rather different conceptual frame- 
work, handles the problem of conflict and conflict resolution in a somew hat simi- 
lar manner. See also Brown (1957) and Miller (1958). Fora discussion of the 
question of the permanence of the inhibitory affect produced by punishment, see 
Skinner (1938), Fstes (1944), Lichtenstein (1950a, 1950b), and Decese (1952, pp- 
rig feo. 
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It is not vet certain how the resolution of this paradox really goes, 
is this. Bull & Strongin have already noted (Section 
and threat are intimately related; and it may 
he first reaction to deprivation (loss of accus- 
hen this ezergizing emotion 
second emotion supervenes: 


but one possibility 
IV) that frustration 
be that, typically, t 
tomed satisfaction) is anger and that w 
fails of its purpose (if it fails), then a 
discouragement, despair, depression. There- 
fore, only in the latter phase of this sequence would we think of 
counterconditioning (and extinction) as occurring and of the origi- 
nal habit strength (hope) being dissipated (or “worked through,” as 
psychoanalytic writers sometimes say in connection with mourning 
reactions). At least, such a biphasic interpretation of “extinction” 
(frustration) would be consistent with the experimental findings of 
Bernstein, as shown graphically in Fig. 11-7. Here we see that a brief 


namely, disappointment, 


Mean trials at or below 
over-all median 


6 8 10 


0 2 4 
Duration of block (sec.) during prior extinction 


0 


a runway response (in rats) as a function 


ance to extinction of a 
sar a special delay (frustration) compartment 


Fig. 11-7. Resis 
of length of time subjects were held in 


(Bernstein, 1957. p-93)- 


(two-second) detainment of the subject in the course of running a 
simple maze increases resistance to extinction but that a longer block- 
age (of eight seconds) reduces it, i.e., facilitates the extinction process. 
The results of this experiment offer evidence for the occurrence of an 
inereque à se cones ES. and resistance to extinction W ith the addition of 
ea extinction Both of these findings are consistent with 
ance € j > LANE, : 
following from a frustration-drive interpretation. A re- 
s gor and resistance to extinction occurred when the 


eded a critical point (Bernstein, 1957, p- 93)- 


need not be elaborated, the precise 


blocking in avoid 

* expectations f 1 
versal in both response v! 
duration of blocking exce 
for reasons which : 
rnstein results is not unequivocal; and it may be 
rily support the particular biphasic theory of 
“This theory is nevertheless a plausible pos- 
e specifically explored in the future. 


However, 
Meaning of the Be 
that they do not necessa 
frustration here suggested. 


sibility—and something t9 be mor 
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IX. Two Recent Studies Described and Evaluated 


In 1939 Razran published a detailed analysis of the ee 
facts and theories of extinction. In 1956, in a paper entitlec br a 
tion Re-examined and Re-analyzed: A New I heory, this SEES m ea 
on the basis of quite different experimental studies, has arrives 5 "e 
interpretation of extinction which seems to be pointing in muc 
same direction as does the present analysis, ; s , ae 

Razran begins by noting that, of late, American investigators. ha 
been more interested in the study of ins rumental or operant bebavioi 
than in what he calls "classical conditioning," by which he means 
conditioning involving external, rather than 
stimuli. This trend, savs Razran 
Opinion, the 


response-correlated, 
| ds somewhat unfortunate since, in his 
acquisition and extinction. of behavior can be usi 
stood in the ultimate sense only in terms of simple, or “classical, 
conditioning, Razran's argument is built around three main proposi- 
tions, the second of which 


^ ES (n Te 
“is the most novel and crucial one. 
runs as follows: 


The automatic 
result of the 
(feedback € 
the CR situ 
the evoke 


deconditioning in the carly stage of extinction is a direct 
loss of the interceptive and the proprioceptive conditional satan 
S's) which in the original conditioning were an integral part o 
ation and which w hen the unconditional stimulus is withheld and 
d reaction is reduced cease to be present (p. 45). 

It will not be possible te 
tion means to Ra; 
terminology alone, 
vised two-factor 


? give a detailed account of what this proposi- 
'an; but it will be sufficiently clear, from the 
that his thinking is close to that inherent in rc- 
theory. The general nature of this affinity. is made 
particularly: evident in the concluding section of Razran’s paper. En- 
titled “Final Word,” it reads thus: 


he testability (verifiability ) of the 
proprioceptive stimuli held by the 
and counterconditioning in extinct 
of intervening variables, They are obsery ables! 
last 15 vears the Russians have cle 
teroceptive and proprioceptive (particularly the former ) and 
their interaction with typical exteroceptive conditioning. And while, as far as 
the writer is aware, no experiment has ev er been specifically devoted to a 
coniparison of the CR states of the vis 


> ; sive eee 
cera during alimentary e 
and during alimentary extinction of conditioning, there is no reason why such 


in esperiment could not be undertaken, Moreover, one could of course com- 


pare extinction of, let us sav, alimentary: conditioning in normal animals and 


new theory 
theory to be the | 
ton are by no me: 


The interoceptive and 
is of the deconditioning 
ans postulated as any kind 
As already mentioned, in the 
ated both the existence of in- 


arly demonstr 
conditioning 
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et 3 ith decafferented viscera. The isolation of proprioceptive stimu- 
presents more of à problem, no doubt, but could also be mastered. 

E Extinction of operant conditioning. It has always been the opinion of 
the writer that a thorough analysis of primary variables in operant condition- 
ing | habit or trial-and-error learning | is difficult, if not impossible, without 
a prior analysis of such v: iables in classical conditioning. And it is only 
such a "prior analysis” w hich this article offers with respect to extinction 
of operant conditioning, even though a number of facts presented here have 
been directly verified, as is known, in operant CR experiments, and the the- 
Ory, too, is not alien to operant CR thinking. More than that, the writer 
is by no means ready to accord priority to operant over classical condition- 
Ing with respect to theoretical and methodological—and even practical fun- 
damentals of learning. The contrary appears to him to be true, and he indeed 
deplores the decline of cla sical CR experimentation in this country. Cer- 
tainly, such extreme views as expressed recently in this Journal by one author 
(Smith, 1954). that "conditioning [meaning € sical conditioning] is an 
artifact” are indefensible—and patricidal. They could have been put for- 
ward only in total unawarenes of the almost 500 well-controlled experi- 


ments in classical conditioning for most of which the author's alternate “rein- 
forcement" is a very contorted al or impossible explanation. 


e and unnatur 
Frue, a large number of these. classical CR experiments are, unfortunately, 
aders. Yet 


unknown to American rea they do exist (pp- 49-50). 


ioted, Razran seems to be espousing a 
revised two-factor theory. “Operant 
is reducible, he conjectures, to 
"classical conditioning; of habits is a matter of “de- 
conditioning and counterconditioning." All learning thus reduces, as 
in "vti-faerr theory, to sign learning and all extinction to a reversal of 
; č is vet another instance of in- 


Here, it seems, 
common point of view 1n these matters 


In the second. pa agraph qt 
position very close to that of 
ic. habit formation. 


conditioning," 
and extinction 


the meanings of signs. 
dependent convergence upon a 


from quite dissimilar starting . j 
But we must now turn to what appears to be evidence against the 


general position taken in this chapter. In a recent paper entitled “An 
Experimental Validation of Conditioned Inhibition,” Calvin and co- 


Workers (1956) present experimental data of a quite remarkable kind. 
rats arc given 30 runs per day na 10-foot 


rewarded. with food, they will 
reward and persistent 


points. 


They report that if hungry 
straightaway alley and consistently 3 
eventually “extinguish.” consistent 
hunger! d M 


despite 


as, & Neisser published a paper in which 
re would have to be logically pre- 
ill's postulates; and since the out- 
nse, they used the prediction as 
c Hullian position. They said: 


an, Nachmi 


In 1954. Gleitim 
an outcon 


they. showed that such 


dicted on the basis of ce 
to common Ne 


rtain of Ht 


come was so contrary : 
an indication of thc unsoundness of th 
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ich c: > ed for 
Necessarily, then, there is no learned act which can ud RUM 
BE rine; d 7 iti 3 ss of reinforcement—mus 
; HS very repetition—regardless o x À 
any length of time; i very tit egal s ber aN Rte 
1 1 7 i This prediction is at odds w TN g 
ad to its eventual elimination. i: th es E 
ie € about the course of learning. The phenomenon of inhibition ^ 
wek a E g : as 4 Mm 
D (Hull, 1943) occurs only under quite special condition € 
: i 1 E. 4 1 ~ . C 
hardly begins to do justice to this deduction. The learning curv aus’ r Max 
ü . . p v 
to zero regardless of the spacing of trials, and must do so in the e 
nber i eri <tincti Ty has reache 
number of trials required for experimental extinction after aj has re ze 
asymptote, One does not have to refer to experimental studies to demonstri 
the fallacy of this prediction (p. 30). 


A ; : kely 
Yet Calvin et al. report dramatic confirmation of this unlikely 
prediction. They say: 


The fact that all of the Ss ceased to run even though reinforced on euan 
trial stands as a striking confirmation of the S—R reinforcement theor 
extinction. Certainly the analysis made by Gleitman, Nachmias, and Ner A 
was a logical one. It was good "common sense." However the beauty of ee 
inforcement theory as elaborated by Spence, Hull, Miller, ete., is its pue 
to make predictions contrary to common sense. Although the presen 
authors are well aware that current reinforcement theory is far from per [een 
it seems to us that as long as it can make predictions of the kind which are 


validated by the present experiment it will continue to be the dominant theory 
in the field of learning (p. 4). 


The experiment in question h 
this level of exposition the 
ever, certain seemingly 


as already been described briefly. At 
results are striking indeed. "There are, how- 
minor aspects of the experiment which can 
hardly be without significance. The first is that after the maze runs, the 
subjects were returned to their living cages 
feeding which supplemented the food obtained as a result of the pas 
formance in the maze. In other words, the rats did not “make their 
living” exclusively by maze running. Indeed, 
perfectly well, or even better, without running the maze at all: if they 
loitered or refused to run they were picked up and carried to the home 
cage, where they received ail the food they wanted! So why “run”? 
Why not simply wait instead of work? ` 

The nonrunning (“extinetion” 
fore, a perfectly logical one. One 
would be “intelligent” 


and were there given a 


they could get along 


) solution to the problem was, there- 


may, however, question whether rats 
enough to hit upon it. 
a "motive" for giving up the work 
solution is clear. The authors say: 


That they certainly had 
solution and adopting the waiting 


In closing we would like to Mention one 


2.550 trials. Fach trial meant that the S h 
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little short of five miles. During this time the E was ac- 
n the alley. Since the E had to walk 


25.500 feet or a 
companying the S on his daily trips dow 
both to and from the goal box, he walked approximately 10 miles. Intro- 
spective reports indicate that a great deal of conditioned inhibition was 
accumulating, and in some cases it was touch and go as to who would 
extinguish first—the S or the E (p. 6). 

ave been carried out with essentially 


Many other maze experiments h 
limited to one or two hundred 


similar procedures, but they usually are 

trials, Here as many as 2,550 trials, or more, might be involved (before 
the subjects extinguished”), so there would be ample opportunity 
for learning of an unusual and more difficult kind than is ordinarily 
u With so much experience, the subjects might 
had a choice between (a) running the m 
al. Morcover, a rather special 
avor just such learning. The 


scen in maze studies. 
well discover that they 
30 times and (b) just wai 
feature of the procedure 


ting for their me 
would seem to f 


authors sav: 
goal box [i.e.n failed to negotiate the maze] 
e ] box. If this occurred on two 
until the following day (p. 5 


If an S failed to enter the l 
after five minutes it was placed in the goa 
successive trials the series was discontinued 


italics added). 
an animal (for whatever reason) delayed 


at it would be carried to the goal box 
to not running; and, more than 


In other words, whenever 
it found th 
ince conducive 
to run, it would th 


in running the maze, 
—certainly a circumst: 
this, if it twice refused 


cage and given free and f n UU 
animal raj for the food obtainable at the end of the maze, it had to 


make 3o such trips before getting anything like a full feeding; whereas, 
if the animal, on only two trials, simply “sat tight" and w aited (for 
a maximum of 10 minutes), it would get a full fecding "for nothing. 
In the Calvin experiment. therc were thus two presione bine y 
the animals’ hunger problem: Q) running thie ans Apote w 
(2) simply waiting a maximum of 1o minuces and being netifEnes to 
$ x food present. Viewed in this light, common 
ad us to expect precisely the result obtained. 
Certainly it is not surprising that the subjects ail, erecta came to 
prefer the decidedly easier of the two solutions. But we can con- 
fidently predict that had the running solution been the only one 
available to the subjects (oF if there had been a long delay after return 


to the home cage before feeding occurred), no such “extinction 


en be returned to the living 


ull access to food, without "work." If the 


the living cage with 
sense" would certainly le 


would have occurred. 


Onlvdn a gere speci: aning habit" here involved 


a] sense can the rur 
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i i i i at i v habit strength, but 
be said to have extinguished. It was not that it lost ha it str ers 
Shabir’ dm - EN tion wet x 
rather that another "habit" got stronger. The extinction deus 
strumental response is normally the decrement in such a respons 


which occurs as a result of its repcated performance, without ss 
expected reinforcement. Keeping the reinforcement present : Ne 
providing an easier alternative wav of obtaining it is pretty clearly a 
phenomenon of a different order (see Deese's 1952 analysis, pp. 117 ff. 
of some of Skinner's and Estes! carly work on punishment). 

We thus end this section with arguments and evidence from Raz- 
ran's paper actively supporting the general view put forward carlier 
in this chapter; and the seemingly 
Calvin et al. lose their paradoxical qu 
circumstances are taken into 


paradoxical results reported. by 
alities when all the experimental 
account (cf. Dinsmoor, 1956) 


X. Kendrick's Repetition of the Calvin Experiment 


After this book had gone to pr 


Kendrick (1958) reported a RE RES 
tition of the experiment by Calvin et al, with certain modifications, 
which calls for special consideration. The rational of this replication 
is given by Kendrick as follows: 


The procedure in this | Ca 


lvin's] experiment was criticized by Dinsmoor 
(1956) who pointed out seve 


ral reasons other than the Cileitman prediction, 
why the Ss may have stopped Tunning, the most pertinent of which was 
that Calvin may have inadvertently: trained the Ss not to run, because when 
any S refused to run it was picked up and placed in the goal box with the 
reward present, thus rewarding the nonrunning behavior, 

Tt was therefore decided to repeat Calvin's experiment in a modified form, 
with more stringent nonrunning criteria exercised (p. 313). 


The apparatus used by Kendrick was a 1o-foot straight runway; 
and, for reasons not explained, the subjects (laboratory rats) were 
motivated by thirst rather ger (as in the Calvin experi- 
ment). Then, after = dle Ss were treated as fol- 
lows: 


than by hun 
preliminary: training 


Each S was placed in the starting 


i box facing the door, the door was 
reinforceme 


opened and S ran to the nt dish. Immediately § finished drinking, 
it was taken out of the goal box and Tun again. This was repeated for 30 
trials for each S. The reinforcement per trial was 5, cc. so that if S com- 
pleted the 3o trials it received 7.5 ce. of w ater. Three hours after the last 
trial of each S a further drinking period ad libitim of 1, hr. was allowed. The 
Ss were thus tested vader approximately: 20 hr. water. deprivation. 
The criterion for extinction was that on three consecutive days S must 
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isecutively within the first five trials. A 


refuse to run for water twice cor 
ime clock within 5 


refusal was detined as S's failing to stop the running t 
toii: te cing ae After the first refus: S was momentarily 
fee cine: we starting »ox and then put bs k in again and the door opened. 
acre > : i E two consecutive refusals during any part of a day's testing, 
5 g was ¢ iscontinued for that day, but was not counted as an extinction 
day. On no occasion was 5 lifted out of the runway and placed in the goal 


ox for reinforcement if 1t refused to run (p. 314)- 


„after a period of about 33 days (on 


By the criteria here stipulated 
water" (p. 318)! This, 


the average), “all Ss refused to run for their 
result—but what does it mean? Certainly one 
source of weakness in the Calvin experiment has been eliminated: the 
d to the goal box when they refused to go there 
But other ambiguities remain, the principal one 
running the maze was not the only way 
It is true that, by running, the 
but if they refused to run, 
isadvantage 


surely, is a dramatic 


rats were wot carric 
under their own power. 
Of Which is that here, again, 
Of getting a 24-hour supply 
rats used in this experiment go 
they still got all the water they could 
‘Thus the expe 


of water. 
t it sooner; 
| drink—the only d 
rimental situation was still very 
an organism refuses to work for 
t to it (nor vice versa). Hence, 
ak of "inhibition of reinforce- 


being a three-hour wait. 
unlife-like; for if, in a state of nature, 
Its living, its living will not be brough 


it seems forced and misleading to Spe 
ment" or "conditioned inhibition" in such a situation. If animals had 


maze as a means of getting 
hat they would vot become “inhibited.” 
i rganisms im. And if they 
t situation, it must be be- 
overall sense preferable. 


to depend exclusively upon running the 


Water, one can be quite certain t 
predicament, o 
in) in the presen 
as in some 
singular practice of discon- 


In the usual sink-or-swim 
preferred not to "swim" (rt 
ment W 


is the rather 
whenever "any S had two con- 


secutive refusals during any part * s testing. If these two re- 


fusals did not come during the first five trials, they did not con- 
+ "inhibition"; but one cannot escape 


less encouraged "inhibition." 


Cause the alternative adjust 
In the second place, there 
that day 


Unuing the experiment “for 
yf a day 


iterion for 
have neverthe 
that the rats used in this experiment, in ad- 


je runway, trial after trial, also did not like 
and handled by the experimenter. Thus, by not 
d escape from this (presumably disagrec- 
rcinforcement of the nonrunning bc- 


tribute to meeting the cr 
the feeling that they may 
Suppose, for all we know, 
dition to not liking to run d 
being picked up 
running, the animals coul 
able) situation, with resulting 


havi 
qVIOF, 
needs to be further replicated, with cer- 


The experiment obviously 
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tain other variations and controls.!? But if, after all qo nndis 
cautions are taken, the Calvin effect still persists, such a T ae ign 
not invalidate the frustration hypothesis; it would simply ci 
fatigue can also be a factor in bringing about extinction, bi e rà 
advocates of the frustration hypothesis have not at all ee à 
it will be remembered that both the frustration hypothes : ane à 
fatigue hypothesis involve basically the same assumption, namely, es 
a form of counterreinforcement is involved in extinction; and it pre 
sumably does not matter gre 
able anger or by conditional 


atly whether this is provided by condition- 
le fatigue—or sly. as Hull has conceived it. 
Both involve the phenomenon of conflict and both achieve their effects, 
presumably, by counteracting hope. In fact. l RN 
even the rats in the Calvin-Kendrick type of experiment are "frus 
trated” —not, to be sure, by failing to find the accustomed quantum of 
water, but by the factor of effort. But it is perhaps idle to speculate 
further about this matter until further investigation shows whether 
the phenomenon is itself valid beyond doubt. 


s it could be argued that 


XI. The Psychology of Courage and Perseverance 


In a chapter on "experimental neuroses” 
Reflexes, Pavlov (1927) 
ton by his student, Dr. 


in his book Conditioned 
describes the following remarkable observa- 
Eroféeva (1913): 

As was mentioned in the third lecture 
be developed to a most sev 
stimulus which would norm 
which the animal now responds by 
head towards the place where the food appears, licking its lips and produc- 
ing a secretion of typical “alimentary: saliva. In the case which is being 
described the development of this reflex had been started with the use of à 
very weak current, which was gradually increased in strength until finally 


it was extremely powerful. The conditioned alimentary reflex developed in 
this way remained stable for many months (p. 289), ' 


ae , flex can 
» à conditioned alimentary reflex eu 
ere electrical stimulus applied to the skin—a 
ally evoke the inborn defense reaction but to 


. . at s 
an alimentary reaction, turning 1t 


Here the innate, unconditioned reaction to a noxious stimulus 
(shock) is overridden or replaced by the conditioned reflex of saliva- 


tion. In order to achieve this effect, it is Necessary, as Frolov (1937) 


"For example, it would be ir 
ing from the goal box back inte 
would be minimized. It is, 
ment" has apparently 


nteresting to use a 
) the starting box), 
incidentally, notew« 
never bcen reported in 
where the animal goes from one performance 
dling or ourside interference of any kind. 


U-maze (with a doorway lead- 
so that the element of handling 
thy that "inhibition of reinforce- 
an ordinary bar-pressing situation. 
of an act to the next without han- 
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in l avlov and His School, points out, for the subject's hunger to be 
quite intense (often involving loss of “a considerable proportion of 
its weight”). But the effect is evidently quite striking. Of this exper- 


iment Frolov says: 


E he animal stands quietly on the stand and awaits food. It suffices merely 
to switch on the electric current and the dog begins to wag its tail and to 
exhibit a number of obvious "food" reactions, the chief of ‘them being the 
process of secreting saliva. On gradually increasing the strength of the 
current, the food reaction does not diminish. Thus, a conditioned food reflex 


to pain has been established. 

In this case, in Yerofeyev 
switching over of the nervous energy 
to the centre of food movements, the previo 
Fhose movements of the dog which were pr 
pletely: ditferent requirements, for instance opening the mouth in order to 
gnaw the [restraining] straps, are now found to be switched over to 
absolutely: different aims and objects, i.c., for better grasping its food, even 
if the food itself is not yet visible. The entire behaviour of the dog has a 
completely different appe: rance from that shown during the elaboration of 
the reflex. Instead of tenseness of the extremities and vertebral muscles, the 
ation and affability. .- - 
is told by the scientist G. Bohn, that when Sherrington—who 
1 f this experiment in Pavlov's laboratory 
shaviour of the dog, he 


the changed be 
joy with which the Christian martyrs 


's [Eroféeva’s] experiment, we have as it were a 
/ from the centre of defence movements 
us reaction becoming inhibited. 
eviously directed towards com- 


Went to the stake!” (pp. 95-97) 
Although the chain of inference by 


at this remark may involve some leaps wl 
) c good use o 


which Sherrington arrived 
hich we are not prepared 
to take, we can, however, mak f both the experiment and 


Sherrington's comment. The experiment dramatically shows how an 
ul can come to release reactions 


external stimulus that is inherently painf 
Now let us ask what would happen if such a 


ied by the experimenter, were a “barrier” 
eros en route to some desired goal (cf. 
is virtually certain that if a rat first ran 
clectric grill on its way to food, it would 
at higher shock intensities, progres- 
ly stop a naive rat (cf. Brown, 


of hope and “joy.” 
stimulus, instcad of being appl 
Which an organism had to 
Warner, 1928a, 1928b). It 
Over a very weakly energized 
become willing to cross the grill 
sively arrived at, which would abrupt 
: ; at, the stimuli correlated with running 


1955). For the experienced ra 
would be conditioned to fear At also, and more strongly, to the hope 


derived from the food reinforcement; whereas a hungry rat, put into 
i th high shock on the grill, would 


tus for the first time Ww! hes 
) running and would be inhibited rather 


in other words, seems capable of being 


the appar: 
have only fear conditioned t 
than “courageous.” Courage, 
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acquired and to be a function of how the Opposing as Hore 
flict are balanced. If much fear and little hope are associated wi ia 
stimuli which a given action produces, then, with respect ee 
action, the individual is “timid,” “cowardly”; but if there is relativ ea 
little fear and much hope, then the action may be böll geenen 
The principle is, of course, a familiar one in military training: m 
posure to slight danger, followed bv relief and satisfaction, as à 
preparation for facing greater danger (cf. Chapter : Section Il). ee 

To the writer's knowledge only onc psychologist has thus fat 
concerned himself with this problem, experimentally, and this rescarch 
got started by accident. Over a period of several years, K. F. Muen- 
zinger and hi: 


students conducted a series of experiments on discrim- 
ination learning in which it was found that reward and punishment 
did not always operate precisely as they might be expected to. Pre- 
viously it had been taken for granted that one rew: 
response in a discrimination situation 
But Muenzinger 


arded the "correct 
and punished the "incorrect" ‘Ones 
and co-workers found, for example, that sometimes 
punishment of the correct response markedly facilitated this type of 
learning, Naturally enough, this finding precipitated a controversy 
(Wischner, 1947; Mowrer, 19502, Chapter 12), which we shall not 
review. Suffice it to say that the paradoxical findings, rather than be- 
ing a contradiction of our general assumptions about reward and 
punishment, seem to be traceable to certain unperceived artifacts 1n 
the situation, However, thesc findings served the highly useful purpose 
of suggesting to Muenzinger a sort of experimental analogue of courage 
and have prompted him to start thinking about this by no means 
unimportant problem (cf. Prince, 1956) j 
The summary of 
follows: É 


a paper by Muenzinger et al. (1952) reads as 


The purpose of this s 
of electric shock for ce 
manifest itself more re 
come previously. 


tudy was to find out wh 
"rrect choice (in 
adily if the avoid 


ether the accelerating effect 
à non-corrective situation) would 
ance effect of shock had been over- 
White rats were give 
$5 of the trials. They were subsequently 
method to learn a black-w hite discriminat 
correct turns were only slightly poorer tl 
The differences were statistically insignificant. 
Among other Ss that had not receive 
the straightaway those trained with sho 
poorer than those trained with shock 
It is assumed that electric. shock 
functions: (a) producing avoid 


n 120 trials in a straightaway and received shock in 


trained by the non-corrective 
ation. Ss which were shocked for 

k aire 
van those shocked for w rong turn 


d any prior experience with shock ee 
ck for correct turns were significantly 
for w rong turns. 

in discrimination learning has tw? 
ance behavior, and (b) accelerating learning- 
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if the tendency to avoid shock iy overcome prior to training in the discrimina- 
tion box, the accelerating function will manifest itself unequivocally (pp. 
118-119; italics added). 7 

And earlier, in the same paper, we read: 

Te might not be amiss to point out that under the conditions used here the 
rats, even though at first disturbed by shock in the straightaway, overcame 
their avoidance reaction fairly well and were willing eventually to cross à 
£ them a considerable shock. The gradual incr 
in current, no less than the frequent exposure to it (55 times), was probably 
an important factor in overcoming undesirable emotional effects, if such 


effects existed at the beginning, which is probable (p. 118). 


grid which must have given 


does not appear; but in a 


word "courage" 
has indicated his in- 


rofessor. Muenzinger 
avior in question as at least analogous to 
this author (in a paper published 
the explicit use of this term, as 


In the paper cited, the 
personal communication P 
clination to think of the beh 
courage. However, as late as 1957 
jointly with Baxter) objected to 
follows: 


lering fear as a motivator, which in 


Vhere is a common practice of consic yat 
our opinion leads to scrious descriptive and theoretical difficulties. 


Where the senior author ( Muenzinger, 1954). has suggested a clear-cut 
distinction between the concepts of motivation and emotion through the use 
of an all-encompassing frame of reference. If the behavior of the shocked 


animals in sets 3 and 4 in avoiding the allev with the signal "shock coming" 
is due to fear, which is superficially plausible, what shall we call the behavior 
of the animals in sets 1 and 2 w hich chose quite readily the very alley which 
also conveyed the signal “shock coming ? If fear of shock did exist initially 
in the | straightaway |. it must have changed into something else after so runs. 
Vhis "something" in. human Ss we usually call "courage. but nobody, in- 
cluding ourselves, would want to use such a concept In rats. Would it not be 
Consistent not to usc the complementary concept of fear in rats? i c do not 
claim that the records of the groups referred to disprove the existence of 
fear in rats which have been shocked; we mere 


lv suggest that in this case the 
use of the concept of fear is superfluous and perhaps misleading (p. 256). 
of reification in science; but as 


e- 


zes the danger 
of fear is concer 
As regards ] 
define it “opera 
“ But may it not be that courage 


Everyone recogni 
far as the. concept 
eminently safe grounds. 
clear, We can, if we wish, 
ether it really 
ice of fear it 


ned, we now seem to be on 
courage." the situation is less 
tionally,” without raising 
“exists. 
a situations W here it might well be ex- 
ach this problem again, from a 
2. In the meantime, it will be 
of “perseverance,” which is 


the issue as to wh 
Is simply the absel 
pected to be present? We shall appro 


somewhat different angle. in Chapter ! 
instructive to examine the phenomenon 
(cf. Chapter 12. Section VHI). 


clearly akin to “courage” 
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Fear of pain is not, of course, the only barrier that Moe ane 
organisms in their march toward the fulfillment of Jur aes X 3 
wants. Effort may likewise cause them to “give up" in [M dee E 
when just a little more persistence would lead to success. es pose : "i 
a hungry rat were put in a Skinner box, with the lever sean 

that it would deliver a pellet of food only after being pressed sii 
times. In the course of its exploration of the new situation, the m 
would undoubtedly press the bar a few times; but it would certainly 
not "stay with" this response long enough to produce any food. \ oy 
likely it would starve rather than learn to survive in this way. Or 
suppose that in such an apparatus, the subject were rewarded with a 
pellet of food each time it pressed the bar for, let us sav. the first 20 
times and that then the delivery mechanism were set so that it would 
produce food only after 200 bar pressings. Here, again, such habit 
strength as would be built up by the 20 reinforcements would probably 
be exhausted well before the rat had pushed the bar 200 times. But if 
the amount of bar pressing required for 
ually increased, then one would find 
that a rat could very well 
The greater resist 


a pellet of food were grad- 
sas Skinner himself reports ( 1938). 
"make his living" in this wav, hard as it i5. 
ance to extinction generated by periodic rather than 
continuous reinforcement will be discussed in 
ter 12); but it will be at once clear : 
here broaching the psychology of “work,” “sacrifice,” “character. 

Also it will be evident that we are 
which is commonly interpreted 
so-called masochism. Here, 


another context (Chap- 
from the foregoing that we are 


here close to a phenomenon 
as a form of psychopathology, namely 
ostensibly, the individual is rewarded, 
rather than punished, by pain—a circumstance which, if valid, would 
constitute no small problem for the homeostatic conce 
tion and behavior. However, 
ably more apparent than real 
As Brown (1955 


ption of motiva- 
such exceptions to homeostasis are prob- 
and arise from a fallacy of observation. 
has pointed out, all goal-seeking behavior to some 
degree involves a detour "through pain"—be it only the factor of 
effort, apprehension, or the like; and it is only when the “punishment” 
is relatively great and obvious, with the satisfaction subtle or obscures 
that confusion arises. When an individual's goal is known and socially 
approved, no matter how great the “obstacles,” f 
“masochism”: instead we speak of 
tion.” Only, it seems, when the goa 
assuagement of guilt—but none the 
dividual himself, and the pain or 
infer. erroncously, that the 


we do not speak ot 
determination, persistence, "gump- 
| is covert—as, for example, in the 
less real and important to the 1n- 
punishment, oz 


t, are we likely to 
“pleasure principle" is being violated. 
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c guilt is undoubtedly a com- 


Incidentally. the capacity to experienc 
overly simple terms 


and one not to be explained in 
10); but it is none the less potent (at the 
human level), and we must not allow our theorizing about behavior 
arerpretation thereof. A spurious (in- 
can always cause much needless dif- 


plex phenomenon 
(cf. Mow rer, 1960, Chapter 


to be distorted by neglect or misit 
sufficiently analyzed) "example" 
ficulty. 


12 


Generalization, Discrimination, 


and Skill 


In some ways the topics discussed in this chapter arc specialized. 
narrow, technical; but in other ways they are as broad and as signift- 
cant as the psychology of learning itself. Here we encounter problems 
which are at once baffling but intriguing. H 


re, too, we begin to move 
from the more elementary learning processes to processes that ane 
“higher,” more complex. Built, as it is, upon principles delineated n 
preceding chapters, the present discussion provides, in turn, a basis 
for the consideration of mediation, 
in the companion volume (Mowrer, 
the more distinctively hum 


language, and other topics which, 
1960), take us from the animal to 
an level of psychological functioning. 


I. Learning and Generalization 


In one sense, even the most rudiment 


ary type of learning involves 
generalization. As an organism 


discovers, or “learns,” that event A 
(a stimulus or a response ) is regularly followed by event B (a reward 
or a punishment), it may be said to be “generalizing.” Even though 
the learner be a nonverbalizing organism, it is as if its nervous system 
were saving, "A was followed by Donee; X vag followed by B 
twice. A was followed by B three Hes ty OA is anl ys followed 
by B!“ In this way, by a process of induction, the organism general- 
ives. or “extrapolates.” from its past experience to the future. On the 
basis of memory of the past. somewhat paradoxically it anticipates 
what is to come and is thus enabled to behave more adaptively than 
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ordinary learning may be termed 
also Mowrer, 1960, Chapter 8). 
psychologists about this 


it could if it did not learn. Thus. 
once-alszays "generalization? (but see 

There has been some controversy among 
arning (Spence, 1940). Doubting 


progressive, step-by-step aspect of le 
some writers (C.g. Krechevsky, 


that learning is essentially. inductive, 
1038) have taken the position that it is instead a matter of “insight.” 
Elsewhere (Mowrer, 1960. Chapter 6) we s all examine the latter 
detail; but we can anticipate that. discussion by 
is real enough, it is not itself a form 
Therefore, the fact of insight does 
at learning proper is progressive, 
a deductive procedure and thus 
There is, it seems, 


phenomenon in more 
suggesting thar although insight 
of learning, properly speaking. 
TOU gO against the assumption th 
inductive. Insight, as we shall see 
complements the notion that learni 


ng is inductive. 


no real contradiction here. 

But the once-always type of gener 
implied by the term, generalization; it is more 
i comes conditioned to respond ina 


to the fact that if an organism be 
to a given stimulus or situation, 1t will 


in tbe same way, though somewhat 
are "similar to" the 


iginally conditioned 


alization is not what is usually 
commonly used to refer 


given way, let us sav with fear, 
also show a tendency to respond 
less strongly, to otber stimuli or 
stimulus or situation to which the fe 
(cf. Chapter 2, Section VL and Chapter 
utility of such a tendency is obvious. If, 
has a painful, fear-inducing encounter W i 
the buffalo henceforth to look with apprehension, not only upon 
but upon all wolves. It may be that some wolves 


is a kind of over-all biological economy 11 


just suggested. ; 
of mro kinds of 
involved in 
generalization, the tendency of living organ- 
hilar stimuli or situations as if they were the 
with which a given experience has been 


situations which 
r was or 
11, Section ID). The biological 
in a state of nature, à buffalo 
ith a wolf, it well behooves 


this particular wolf, 
are harmless; but there 
generalizing in the manner 

Hence, we may speak 
always generalization, the process 


“generalization”: onee- 
ordinary inductive 


learning; and one-man y 


Isms to react to many simi 


original stimulus or situation 


associated. f i 
There has been much discussion of the shape of the curve for one- 
many generalization, OT nae 
c ep human or 


called. If, for example. cithe , l d à 
Middle-C, i.c., a tone of 265 vibrations per 


lar way to: 
tendency to make the same 


a decreasing 
ag out on each side of, i.e., above and be- 


sent the original, inductive learning 


generalization, as it is more often 
animal subjects are taught to 


react in a particu 
second, they will show 
tones extendit 


response to the 
if we repre 


low, Middle-C. ‘Thus, 
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by means of a rising, negatively accelerated “earning Cee he 
think of this learning generalizing off in both fige eode 
stimulus involved in the original learning. A COME ARENDS m soe a 
this relationship between "regular learning" and stimulus a feels 
tion” is shown in Fig. 12-1. There may be some errors us t i e T 
of this figure, but it is presumed to represent, in a i * p 
relationship between the two forms of "generalization" under discus 
sion here. 


Response 


Fig. 12-1. Three-dimension 
the curve of inductive lear 
the curve. 


al graph showing 
ning and the gene 


the hypothetical relationship betweer 
ralization gradients radiating off fron 


So far as the writer is aware, no empirically derived set of curves 
such as shown in Fig. 12-1 has ever been published. However, since 
the original copy for Fig. 12-1 was drawn, Guttman & Kalish (1956) 
have published the family: of curves shown in Fig. 12-2. These show 
the extent of generalization as a function of 

different stages of response extinction. The picture is very similar to 
that postulated above as representing generalization as a function of the 
level of response strength during the course of acquisition. The general 


response strength at foul 
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me os 3 ur z Ta ES 
iethodologs used in obtaining the Guttman-Kalish curves 1s discussed 


in the next section. 


Responses 


ex 


ned by Guttman & Kalish (1956. 
wenty-four pigeons served as sub- 
derived, are to be compared 


adients obtair 


Fi À : 
ig. 12-2. Mean generalization gri 
pe 85) at four different stages of extinction, 1 
These curves empirically 


iin Fig. 12-1 


ISSIS this experiment. 
B : 
vith the hypothetical ones showt 


ll. Generalization and Discrimination 
at occurs automatically, on 


lization something th 
-is it, too, subject to 


innate mechanism, or 
‘This much is well known, that the 


genet ization can be narrowed through 
rion already given, our subjects have been 
C with fear, we can reduce their tendency 
tones by (a) presenting those tones many 
cinforcement or (b) by having those other 
ntal reinforcement (scc Chapter 11). In 
an be established, as a result. of 
ly to Middle-C and neutrally 
the shape and magnitude 
D generalization gradient extending Out 
has been radically restricted as a result 
more extended gradient is in some sense 


Is stimulus gener 
the basis of some sort of 
Modification through experience: 
Spread or extent of such 
learning. If, in the illustra 
taught to react to Middle- 
i react fearfully to other 
times without incremental r 


tones followed by decreme 
^p discrimination € 
t with fear on 
earby tones. Herc 


this way a clear-cut 
Which the subject will reae 
GE OVE positively to other n 
of the double ( bidircctiona 
from the original training tone 


of learning, so if the originally 
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i i rersibly fixe hough, as we 
innately given, it is obviously not irrev ersibly fixed. Alt R 
ana g ; \ r irre ME den 
: cen, it is to the advantage of living organisms to genera T 
i ded i ituati 3 er similar situations, 
they have learned in one situation to other similar situ 
also desir 


able that, where further experience w Mandel hae ur E 
of "correcting" this tendency so that behavior conforms as exactly 
as possible to existing realities. M - S T 
Having thus seen the possibility of restricting the FARBE up 
generalization bv means of discrimination, onc WORDS 1 APES 
any w ay in which its range might be extended. Rather g T sa 
this could be done, in the illustration given, by also negatively 
forcing tones other than Middle- 
first found that Middle-C was dar 


i ; ; an 
If, in this procedure, the subje 
— Eg "€ anc 
igerous, then that the G nh 3 
t y re the G and the © 

the F below were dangerous, and that the C above the G anc 


> ieee EE -ertainly have 
below the F were also dangerous, the subject would certainly 
a tendency to “infer” 


equally so. 
below, or 


that all tones are dangerous and probably 
Hence, the subject might react to any tone, v hether arie 
in between those actually reinforced, quite as. vigorously 
as to the reinforced ones, Here we 
to, if not identical with, 
(Osgood, 1953; Hilgard 


are dealing with something similar 
what is commonly called concept berg 
+ 1956). An auditory stimulus of Oe os. 
vibrations per second is certainly very different from one of Ey xia 
but we do apply the term, tone, to cach of them and to all the intei PS i 
ing frequencies; and if an organism has the "concept" of tones, it ii 
be a relatively simple matter to conclude that "tones" are Pici 
and thus to react with equal vigor both to those that have been spe- 
cifically reinforced and to those that have 
mediation, as developed in Mowrer, 
we have, in a very 


not been (see concept of 
` 2 SG 
1960, Chapters 2, 6, and 7). He 


real sense, the antithesis of 
In the interests of brevity, 


the matter of gener 
Before concluding 


discrimination. ; 

the foregoing discussion has dealt with 
and discrimination in rather global terms. 
this section, it will be helpful to review a series of 
very precise experiments recently reported by Dr. Norman Guttman 
and colleagues. A good many vears ago, p, p, 
a useful method for studving di 
example, a pigeon 
panel or “key” 


alization 


M j Scand 
Skinner ( 1938) suggeste 
imi A 2 : Pet [. for 
serimination in animal subjects. If. f 
is hungry it will readil 


" i c 
v learn. to peck at a littl 
as a means of obt 


aining bits of food. Moreover, if a 
may be said to re 
As Selye (1956) 
fever 
sponse to smoke and certain other ga 
only when they “generalize” 
harmless substances such as pla 


“Allergies” or sensitivities present failures of "discrimin 


Y. i A reactions 
tion" at the physiological level, has pointed out, the reactie 


involved, for example, in “hay 


Na PME 

TC highly useful when they occur s l 
= A ca 

ases and irritants: 

too broadly 


nt pollens. 


they become patholog n 
3 à i 
and occur in response to biologically 
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Sw dieere ssi ^ = 
u tdiserininative stimulus or SP) GE some-sort P introduced, the 


only when the cue is present, 


bir i Y 
rd will soon learn to peck at the “key” 
espond “on cue.” Using 


ut AN ie MENU 
at when it is absent (82); Le. 1t learns to T 
6) have been able to derive 


thi NN " e 
s basic procedure, Guttman & Kalish (195 
gradients for different 


fro ~ * e P 
m the pigeon very pretty generalization 


re i F . a x. ok 
egions of the color spectrum, 25 shown in Fig. 12-3. This was done 
50. 
200} 
g 150} 
5 | 
2 
E 100} 
| 
50 
1 A 
E i 
0 = m ne -22> 
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Wavelength ( mu) 


Guttman & Kalish (1956, 


adients obtained by 
‘The gradients for the 


Fi AM 
ig. 12-3. Mean generalization gr 
(six to the group). 


E 83) from four groups of pigeons 
ndividual pigeons were very similar. 
c spectral color, c.g, green 


when som 
ignal window and not to 


c milkglass 5 
1224) and then exploring the 


ld be reacted to as if they 
account of their re- 


by teaching the bird to peck 
pon th 
as dark (Fig. 
ther hues wou 
In a recent popular 


(550 mp), was projected u 


peck when the window W 
extent to which lights of o 
Were the cue for pecking: 
Searches, Guttman & Kalish (1958) say? 

We first train the pigeon te peck at light of a certain W avelength, pre- 
sented in the form of an illuminatec disk. T he food reward for pecking the 
disk is given nor for each response but on an irregular schedule, controlled 
Dy-ariadtomatie device which gives food for onc response In each minute on 
the average, The main reason for this system 5 that we do not want the bird 
to "expect" d reward for every response, because that would interfere with 
the later generalization rests. Paradoxically, the irregular reward method 
produces an extremely eager response by the pigeon. After a few hours oF 
training the bird pec : E - at the rate of 4,000 to 6,000 times per hour! 

hat is especially importa the response persists strongly 


When we stop giving the reward. [ yntinues to peck for several 
hours, at E iv declining rate- During this period we can carry out extensive 


PRI 2 
Beneralization tests. 
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A test consists in presenting the pigeon with monochromatic light T 
10 or 12 wavelengths other than the one to which it was trained to zespo: 
The test stimuli are distributed over a considerable range of the color w 
trum. If, for instance, the bird was trained to a green at the w avelengt due 
5,500 Angstrom units, it may be tested on wavelengths ranging from a 
Angstroms (blue) to 6,200 Angstroms (orange). These are presentec 

random order. The whole series of w avelengths is repeated a dozen ee 
each time in a different order. The bird’s response to cach disk is measured 
by its rate of pecking (pp. 78-79). 


Fig. 12-4. A portion of the 
in the study of color gener 
could be illuminated with a 


apparatus used by ( 
alization in the 


ny desired colo, 


juttman & Kalish (1958, p. 78) 
pigeon. The “cue w indow," at the right, 
r of light or allowed to remain unlighted. 


Many earlier attempts have been made to 


ascertain the exact shape 
of the curve of sensory gener 


alization, but numerous difficulties have 
always been encountered w hich make one somewhat less than. con- 
fident that the results in any given experiment 
curve. Wickens, Schroder, & Snide 
variables that may distort the results o 


represent the "truc 
(1954) have discussed possible 
btained in this type of rescarch; 


and. on the basis of experiments in Which special care was taken to 
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avoid artifacts, they conclude “that the gradient of primary stimulus 
generalization is actually bell-shaped” (p. 56). The results obtained 
by Guttman & Kalish strongly support this assumption. 

In some ways. it is little short of astonishing that the pigeon should 


"know so much" as it apparently does about the color spectrum. 
That a bird should ditferentiate between a pure green and a blue-green 
llow-green on the other is perhaps not so sur- 
able that it should "know" so perfectly 
that red (640 my) is “farther” away from green than is yellow 
(580 mg) and that dark blue (420 mye) Is farther away, in the opposite 
direction, than is light blue (460 my). Human beings, in the course 
of their socialization, nearly always receive at least a limited amount 
of training which helps them perceive tones with rapid frequencies 
as "high" and tones with slower frequencies as "low" (see earlier 
example). But one cannot see anything in the experience of a pigeon 
similar ability to order colors so accurately 
The most reasonable inference at present is 
ay innately given; vet, if innate, this 


on the one side and a ve 


prising; but it does seem rem: 


thar would give it 
according to wavelength. 
F is in some W 
is subject to modification through train- 


that this capacity 
capacity (as already noted) 


if given discrimination training, can become capable of 


ing. Pigeons, 
differentiating quite sharply | 


another which differs from 1t 
crons. In other words. the "innate" tendency toward color generaliza- 


corrected so that the drop in response rate, as one moves 
forced stimulus out to unreinforced stimuli, is precipitous 


)etween one monochromatic color and 
in wavelength by only 2 or 3 millimi- 


tion can be 
from the rcin 
rather than gradual. 


Furthermore, Honig (1956 1 j 
by means of special tri 
) generalization gradient broader. 


), working under the direction of Gutt- 
ining, it is also possible 
as well as 
if pigeons 


man, has shown that, 
to make the "innate? color 
narrower. As shown by th 
receive reinforcement with 
equally. well to stimulation 


one hue, let us say. pure green (550 
zes in the way shown by the solid curve in Fig. 12-5- 


much the same pattern of generalization with both pos- 
reinforcement (Chapter 11). although the two 
irror images of each other. 

of course, be possible to modify 


c broken curve in Fig. 12 
arious hues, they respond about 


cues of v. 
from any part of the spectrum. Then, if 
ma), is extinguished, the extinc- 


tion generali 
Thus we see 
itive and negative 
bsolute terms, are n 
it would, 
rve, just as it is possible to modify the 


patterns, in al 

With further training. 
the extinction. cu 
d be equivalent to one’s being able to reproduce 
al knowledge or training. 


(“sharpen”) 


* At the human lev el, this woul $ 
ithout speci 


the full color spectrum Ww! 
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er curve for positive learning. If, for rie sd ee 
Son of green, a pigeon repeatedly found that ie T aed 
jacent edlür of blue-green was "positive," the soif: P pns 
ruptly from green to blue-green, rather than gradually as s 

* 19. Z5. i r 1 " 

i Es is obvivasly possible to use discrimination learning for ceo 
two purposes: (1) to determine the range and aun ha cr ne 
receptor system, or (2) to investigate the learning process itself. 

are here interested mainly in the latter, 


Ill. Conditioning and Discrimination 
not Easily Distinguished 


Let us suppose that a laboratory 
compartment, the floor of which is 
tric shock can be administere 
buzzer will sound for 


rat has been placed in a small 
a metal grill through which elec- 
d to the rat's feet. Now periodically a 

five seconds, at the end of which time a shock 
will come on the grill and remain on until the rat, in the course of trial- 
and-error behavior, le; 


aps into the air. If, when the buzze 
the rat leaps in response to it alone, 


period, then the buz; 
In short, the procedure 
the danger signal 


is presented, 
within the five-second warning 
is terminated and the shock is not pr sented. 
is doubly “instrumental”: the leap terminates 
and also prevents the shock from occurring if 1t 
occurs to the danger signal alone (scc Chapter 2 and 3). 
During the course of the 

two or three leaps from 
buzzer alone; 


first 10 trials, one may get an average of 
à group of rats, treated in this manner, to the 
but one will note 
number of leaps in the 
shock is on, 


an equal or perhaps even gucanet: 
intertrial interval, when neither buzzer nor 
How does this happen? After 
combination 
two, it will be apparent th 
of whether the buzzer is 


à rat has received the 
in the experimental situation a time o 
at the rat is afr: 


buzzer-shock 


aid of the situation regardless 
present, or not. In other words, in the begin- 
ning the rat becomes quickly. conditioned to react with fear to the 
experimental. situation, as such, and does not distinguish sharply be- 
tween situation-with-buzzer and situation-without-it. However, if the 
subject is repeatedly put back into the apparatus 
10 trials each dav, we Observe th 
with-buzzer increases 


and given, let us sav. 
at the incidence of leaps to situation- 
and the incidence of le 
buzzer decreases. This tendency: for the * 
pear and for the responses to the 


aps to situation-without- 
"interval" responses to disap- 
danger signal to become more reliable 
is shown in Fig. 12-6, Although the rats become quickly conditioned 
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they gradually learn that the situation 
the buzzer is present, at other times 
Although a curve such as the ascend- 
a record of conditioned- 


to fear the situation as a whole, 
Is really dangerous only w hen 
perfectly safe (ef. Section X). 
ing one in Fig. 12-6 is often presented as 
apparent that such a curve also 


learning and does not by any 
As already indicated, con- 
, as 


response learning. it will thus be 
Importantly involves discrimination 
Means portray conditioning in pure form. 

ditioning, in the sense of fear learning, occurs in one or two trials 
can be demonstrated by the fact that a rat, once (or twice) shocked in 
a given piece of apparatus, will react thercafter very differently if again 
a week, or even many weeks after- 
ence therein. In other words, the 
two encounters with 


placed in the apparatus a day, 
wards, without any intervening experi 


rat will "remember" the shock after only one or 


45 
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) as revealed in a study car- 
as subjects. Actual extinction 
xtinction generalization is 
lors of various hues prior 


or inhibition 
cons serv ed 
The obtained € 
g to CO 


f the figure. 


Fi . . H "dri 
ig. 12-5. Generalization of extinction. ( 
s x ae 
ted out by Honig (1956. p- P Six p 
us OP $79 0. i 
el of respondin 


The lev 
“pascline™ at the top o! 


teok : a 
We place with a stimul 
Sh * 
own by the lower line- 
AEN CI s 
extinction is shown by the 
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it in this situation. Hence, the type of ascending curve shown in 
Fig. 12-6 gives a very inadequate picture of conditioning in pure form. 

Further support for the type of thinking just described is given by 
the two curves shown in Fig. 12-7. Here the danger s 


ignal was a dis- 
tinct, momentary "tap" caused bv the clapper of an electric bell 
striking the bell just once, the bell immediately thereafter being 


10 


Fig. 12-6. Curves reproduced 
from Mowrer and Lamorcaux 
(1951, p. 2047, showing the in- 


cidence of "avoidance" jump- 


Responses 
< 


ing responses (ascending 
curve) and the incidence of 
“interval” jumping responses 
(descending curve). 


Days 10 


"damped." If, within five scconds after the 
the subjects did not leap into the air, they received electric. shock, 
Just as did the animals whose performance is shown in Fig. 12-6. But 
during this five-second danger period the only basis for diatinwtisting 
benyen it and the safe period, between trials, was the trace of the 
tap (Chapter to). It would, the experimenters reasoned, be more 
difficult under 


'» we these. conditions for the subjects to distinguish the 
danger period 3 ^ safe m : p z ck 
ger T d and the safe period than it would be for subjects in à 
situation where the d 


anor a anger period was demarcated by a continuous 
uzzer. The fact th: to qu 'etti i i 

biez € fact that the rats getting the discrete “tap” made more 
interval responses and fewer 


occurrence of this stimulus, 


clearly that the discrim; avoidance responses seems to indicate 
eariy thi `r et $ a 

C y that the discrimination had indeed been rendered more difficult 
for them. 

ae ia findings that could be cited make it sufficiently 
pn ua s UTER m enteria commonly used as a measure of con- 
ditioning is really heavily contaminated with 
Conditioning seems to occur 
situation; and only gradually 


i discrimination learning. 
first, and. very quickly. to. the. total 
does the subject then differentiate out 
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of that tota 


bination of elements that is truly significant, sp 


A small boy's mother put his new dog 
the back door. The dog tried immediately 
with the and since he was not 
shouts of “No, no!” the dog was switched 


was “willing” to remain outside 


woman; 


the boy reportec 
the house with him 
so conditioned as to fear going i 
regardless of such scemingly trivial details 
told “No” or “Come on in." Eventually, 
crimination was made; but conditioning, 
is obviously much more rapid. 

In the foregoing we have 
occurs when a rat learns to 
shocked in a given piece 
that occurs when the rat le: i 
‘That the distinction 


aic 


arns to be af 


buzzer is presented. 
clear-cut is indicated 


]. with some puzzlement, tl 


through the back door 
ato the house through the rear door, 


as to whether 
of course, this kind of dis- 
in the gross, over-all sense, 


implied that it 
be afraid as a 


of apparatus and that it 
1 in this setting only when a 


] by the fact that the rat 
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| stimulus complex the specific stimulus element or com- 


ccifically meaningful. 
out of the house through 
to go back into the house 
deterred by the woman's 
a couple of times, until he 


and let the woman enter alone. Later 
at his dog would not enter 


Clearly. the dog had been 


he had been 


is conditioning which 
result of having been 
is discrimination 
even here is not entirely 
“conditions” so quickly 


15 


showing the 
res 


Fig. 12-7. Curves 
incidence of 
sponses (below) 
dence of interval 
(above) in a situati 
the danger and safety in tl 
perimental situation are 
tionally made more difficult to 
discriminate ( Mow rer and Lam- 
205). 


0 


avoidance 

and the inci- 
responses 
on in which 
je cx- 


Responses 


inten- 


Oreaux, 1951, p- 


Days 
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fr > $ is situa- 
to the apparatus-as-a-whole only because, from the outset. this wo 
. “ee E . . »" ae A " Nt H ‘h the 
tion is already “discriminated” from all other situations in w s 
rat has ever been. Usually, in discussing discrimination, wc speak 


cue; and in discussing conditioning, we refer to none 
stimulus as a signal. Yet these terms are constantly emake) : 
example, Miller (1951b) says: "A learnable drive or rewarc Maes 
conditioned fear or hope, in our terms] is one that can be acquirec d 
a previously ineffective cue as a result of learning (p. 436; Italics 
added). And Gentry, a good many ye as 
visual discrimination spoke of a slight change in illumination as : 
signal for the occurrence of an avoidancc responsc. Although, follow- 
ing Gentry, many other investiga 
ditioned-r 


ars ago (1934), in a study of 


ators have used the so-called con- 
sponse method of studying discrimination (cf. also AH 
1927, and his co-workers), the impression seems to persist that there 
is a real, albeit subtle, difference between conditioning and discrimina- 
tion. Here we can only conjecture th 
ordinarily used, involves affective 
involves ‘cognitive response; 


at conditioning, as that term Is 
responses whereas discrimination 


but since the latter term will not be given 
explicit consideration until later ( Mowrer. 


1960, sce especially Chapter 
5), this distinction cannot be cl 


aborated here. 


IV. Some Interesting Complications 


Those writers who, in the past, have wished to stress the discriminat- 


ive (“cognitive”) aspect of learning in contrast to the reinforcement 
("conditioning") aspect have sometimes pointed to the fact, long 


known (Pavlov, 1927), that if a laboratory animal learns to make a 
particular response, which is then e 


xtinguished, relearned, extinguished, 


For a discussion of the conditioning-discrimination distinction in somewhat 
different terms, see MeGuigan & Crockett (1958). 


epigrammatically, remarks that "AI learning 
This is w hy theories of discrimination 
Although the last word has certainly 
discrimination and conditioning i 
ent or at least not precisely "i 


See also Kendler (1959) who. 
results from learning to discriminate- 
are considered as so important” (p. 53)» 
not been said on the 
(cf. Section V, 
identical, 
piece of apparatus and does nor re 
he does not discriminate 


relationship between 
the two seem genuinely differ- 
«if a rat is put into a particular 
act with strong fear, 


this situation 
cage. The discrimination of 


For example 


+ this is not to say that 


as different, let us say, from his home 
difference, Perceptual difference, į 
diate; and we may then Specifically condition fe 
it with clectric shock or other tr 


that the rat has also made an c 


nd imme- 
sociating 


i de casy 
ar to this situation Dy z 
aumatic experience. Now we may say, if we like. 
motional "discrimination" between the two places. 
bur this does not imply that perceptual or "cognitive" discrimination did not pre- 
V iously Cxist. 
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nd extinctions tend to occur more and more 
this finding is somewhat difficult for 
However, in 1945, Mowrer & Jones 
and the absence of a primary re- 
quite aside from their rcinforc- 


Cts, successive retrainings a 
rapidly. Superficially at least, 
reinforcement theory to explain. 
pointed out that both the presence 
ward (such as food) can have eve value, 
mg (or frustrating) functions. And Wickens & Miles (1954). in a paper 
entitled "Extinction Changes During a Series of Reinforcement-Extinc- 
HO Sessions" have more recently published particularly good ex- 
perimental evidence on this score (Sce Fig. 12-8), which they interpret 


a 7 š pes 
long much the same lines. They sav: 


in the stimulating [experimental] 
f the first bar-pressing act. 


act and the experimental arrangement, à food 
s, when the bar is confronted on the 


e is no difference 


For the animal, ther 
- until the completion o 


situation on any one dz 
As a consequence of this 
peller is cither received or not Thu 
second trial, traces of reinforcement or nonreinforcement are then present. 
is such that on davs when traces of rein- 
ill be followed by reinforcement, 
onding is not reinforced. Hence, 
basis of traces of reinforce- 


he experimental. procedure 
E are present, further | 

these traces are absent, further resp 
to solve the problem, a discrimination on the 
Ment or nonreinforcement must be learned (pp- 316-317) 


r respc anses W 


— s 
! : 10 15 
Extinction sessions 
Fig. 12-8. Number of response? made on successive ESHGHOUS: with intervening 
Te-aequisition. (Wickens and Miles. 1954+ P: 316). It is interesting to note that up 
à ) had not learned the “meaning 


jects (rats 
The m 
tes that such 


the sub t 
arked decrement in number of 


to " . i 5 
the third extinction. sesstens 
"meaning" bad been 


of = 
H ag the first r 
E juently 


"t r n espons 
nonreward. follow ce p 
indica 


Ntineti á 
Netion responses made subsec 


learned, 
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i iy thier words, primary reinforcement (food) or pepe ccu 
ment following the first trial of an experimental session has, in et ^ 
to its inherent (positive or negative) "motivational si Ps yen 
ter 11), also a cue function, which indicates pica n eigo 
responses during that session will or will not be s edes AN pes 
worth or not worth the trouble of making (see also Lauer & ees 
1957; and Brunswik's 1939 study of repeated reversal of a T-maz 
habit). 


Reinforcement theory has been similarly challenged by the aes 
first reported by Harlow (1949) and confirmed by many others, MM 
Meyer (1951), Warren & Baron (1956), Miles & Meyer (1956), pu 
if monkevs (or other laboratory animals) arc given a scries of oie 
crimination problems they “learn to learn,” le; become progoseoh 
better in their performance even though cach discrimination problem 
or task is distinctive and of equal inherent difficulty, This finding 
forces a long-overdue recognition of some important facts, but it does 
not seem to require any new principles of explanation. The present 
book was first completely written and mimeographed; this EESLI 
was used for teaching purposes and for obtaining the criticisms of 
colleagues; and then cach of the 12 chapters Was revised and new 
typewritten Copy sent to the publishers. 
or three chapters required nx 
chapter was revised in one 
task,” but there was much 


The revision of the first INO 
ore than two weeks apiece, whereas this 
week. Here cach chapter, also, was a IO 
transfer (generalization) of prior learning. 
as regards methods of organizing the task, work routines, and many 
other little “tricks” and conveniences not known at the outset. It 
seems likely, therefore, that Harlow^s monkeys did not so much 


"learn to learn"; instead, they began to transfer the 
from task to task, Which is to say, 


learning tasks (ef, Hebb & * 
10). But Harlow's study is 
reminds us of one of the un 
subjects. Because, in any 
things have to be le i 
behavior in 


"common elements” 
to generalize about discrimination- 
Vhompson, 1954; Mowrer, 1960, Chapter 
none the less important and useful, for it 
avoidable hazards of using "naive" animal 
given learning situ 
arned (or unlearned) 
Which the experimenter 
laboratory studies have undoubtedly 
"slower" and less intelligent. than 
studies dramatic: lly show, 


ation, many incidental 
in addition to the particular 
is specifically interested, our 
often made animals appear much 
an they actually are. As Harlow’s 
when monkeys get their "naivete" out of the 
wav. they are clever as can be. A large part of man's superiority Hn 


"learning? and "problem solving" undoubtedly lies in the fact that iy 
any new situation he brings N-vears of "education" «ith him. This 
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problem will be discussed again, in a di uen. 
Eso Nw: ye discussed again, 1n à ditferent context (Mowrer, 1960, 
irected to an experiment which has 


Fi . ; 
inally, attention should. be d 
y "rhe conceptual background is 


been proposed. by Flakus (1958 
formulated thus: | : 

egards the initially neutral stimulus as having 
t “fear” or “anxiety. According to Mowrer, 
rned according to à stimulus- 
ally neutral stimulus serving as the CS. 
as shock, as the US. This hy- 
properties, but more im- 


Mer ipe of avoidance r 
pa A i capacity to elici 
K lk (a Te isa conditio di 
d B principle, wich the ina 

` primary noxious stimulus such 


»othetic: 1 S 3 ^ 
(ie tical fear response Is endowed wich drive 
ant for the present purposes the covert stimuli produced by the fear 


i pati to be regarded as the directing, oF cue, stimuli for the observ- 
a dance response. This is clear from Mowrers diagrammatic repre- 
ee T n his two-factor theory of avoidance learning, where condition- 
sen nd "problem solving are laid end-to-end. The initially neutral 
iuis us serves as à Cue for the avoidance response only in the sense that 1t 
ads to another stimulus, fear, which i5 capable of evoking this response. 
Mey a conception, although perhaps tenable for the single-response avoid- 
situation, seems to slight the possible cue properties of the initially 


es hi stimulus. If two such stimuli are 5€ parately paired with a common 

ad E stimulus, then presumably they would elicit m an fear response 

Pe Ye same response-produced, stimuli. If it can be shown that 93s capable 

e esponding ditferently to the initially neutral stimuli then it seems obvious 
at they must have a cue function more 


direct. than just eliciting à 
mediating fear. Perhaps different patterns of fear can be established, as Miller 
possibility as never been explored (p. 1). 

animals to make 


on the 


ned response, lea 


Once : 
ce suggested, but this 
ach laboratory 


erent stimuli,” 
"no reason to 


;.). What 
may still 


is to te 
to two diff 
ye. there will be 
yonses” (p. 


ie proposal, quite simply. 
Son different avoidance responses 
sumption that, if this p 
po a hypothetical fea 
MER A na ysis overlooks is the 
Ye motivation common 


roves possil 
a to the res 
at conditioned fcar 
and that they occur 
in effect, two 
“work” 


T commor 
possibilit 
to both responses 
warning signals create, 
y s learn that Ri will 
t not the other way round. 
c in their capacity to 


arouse fear but would differ in their cue properties as far as overt 
behavior is concerned. R, in the presence of S, will be followed by 
reward but not in the presence’ of Se; and similarly, Re n the presence 
of S, will be followed by reward but not in the presence of Sı Thé 
Proposed experiment docs not. jt seems, present nearly so sharp à 
dilemma as at first it might appeal to; the reason this 1s so should be- 


Cc S " 3 MESS * 
"me clearer in Section X of this ch 


y th 


differens 

ferentially because the two 
herein th 
in situatio! 


c subject 
atwo, bu 
would be alik 


ditfere aie 
Terent. situations, V 


in sj i 
situation-one and R 


Md otherwise: the tw? stimuli 


a pter- 
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V. Conditioning and Discrimination: ; 
The Keller-Schoenfeld-Dinsmoor Hypothesis 


Some of the more global reasons for equating the roles of € 
signals have been delineated in the preceding section. We da re 
examine this relationship in a somewhat more specific way. In : 
same year, Keller & Schoenfeld (1950) and Dinsmoor (1950) us 
forward the view that in order for a stimulus to act as a secondary 
reinforcer (tvpe-2), it must also have cue 
it was proposed that any stimulus which ries 
“setting the occasion” for a response, will serve also as a secone AY 
reinforcer. Said differently, the proposal was that every such punc 
has two functions, depending upon whether it precedes or fes 
a particular response, If, for example, a hungry rat is waiting for a 
stimulus which indicates that bar pressing will be effective as a pcr 
of obtaining a pellet of food, that stimulus (SP) is a cue or, more 
precisely, has cuc function. with respect to thc response w hich " 
"releases." On the other hand, if the rat, while w ating for such a 
Cue, discovers that it can produce the cuc, let us say, by licking the fur 
on its left flank, then with respect to this response the stimulus is à 
secondary reinforcer, Therefore, reasoned Dinsmoor: 


function; and, conversely, 
acts as a cuc, in the sense of 


It no longer seems to be necessary to 
of the manner in w hich a discriminative 
gains or loses the power to raise the r 
between the two types of stimulus 
between two categ 
stimulus. This dist 
distribution of the 
organism's repertoi 


provide separate theoretical Rit 

or [secondary | reinforcing stimulus 
ate of response. The previous [gendo 
appears to be reduced to a Dee Rum 
ories of temporal schedule for the administration of e 
Inction is relevant Chiefly ices the dereriminaniot ot TN 
stimulus effects among the variety of responses in the 
Te (p. 471). 

This position is entirely 
revised two-factor theory, In Chapter 3 We saw that if eating on the 
part of a hungry rat is contingent upon its getting a particular cuc. 
the rat will quickly learn to press a bar 
cue (without further primary 
is ordinarily termed “secondary: 
that if eating on the part of a 


congruent with the presuppositions of 


as a means of producing this 
reinforcement), thus exhibiting what 
reinforcement.” In Chapter 7 we saw 

hungry: dog is contingent upon the 
dog's experiencing à particular pattern of kinesthetic and tactile stiniula- 
tion ( produced by the flexing of 


the right foreleg), then this Dind 
will likewise manifest secondary reinforcing properties, as indicaret 


by the dogs developing a Paw-flexing "habit" And elsewhere 
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an important application of this principle 


(Mowrer, 1960. Chapter 3) 
learning will be described. 


in the explanation of language 

In 1950. Schoenfeld. Antonitis, and. Bersh reported an experiment 
convincing in this connection. What these 
rats with a distinc- 
sign or cuc), but 


which seemed particularly 


they presented hungry 
of receiving food (as 4 
vallowing food already given to them. 
ed with drive reduction and would 
acquire secondary- 


investigators did was this: 


tive stimulus, not in advance 
as they were chewing and sv 
Here the stimulus was associat 
(according to Hull's postulate) 
reinforcement powers but, com 
cuon or 
reinforce a bar-pressing response, 
(cues off) the act of cating, 
reinforcing capacity. And 
ngenious experiment 


be expe sted to 
ing after the food was already ob- 


tained, it could have no cue fun properties. It was found that 


such a stimulus did vot subsequently 
ie, unlike a stimulus which precedes 
this stimulus had no demonstrabl 
MeGuigan & Crockett ( 1958) have carried out an it 
on the same problem. which they summarize as follows: 
d such that 


c secondary 


Two groups of rats were trainc a stimulus was established as 
a discriminative stimulus for Group |. but not for Group Il. To accom- 
plish this Group I ran Jin a straight allev with interchangeable goal boxes] 
to either a black or a white end box, only one of which contained food. 
Group ll ran to one of two end boxes of the same color, both of which 
contained food. Performance in a Y maze which had a black and a white 
end box, neither of w hich contained food, was used as the test of secondary 
reinforcement. Two criteria were used: frequency of choice on the first 
trial and on the first 10 trials Performance on the first trial was not an 
bur the second criterion indicated that the stimulus 
ated. with reward became à secondary reinforcer for 
oup Il. Ir was thus concluded that the prior estab- 
criminative stimulus is à necessary condition 
secondary reinforcer (p. 187). 
okett point out, this finding gocs be 
| stimulus needs only to occur $ 
conjunction with a reinforcing 
[to] acquire the power of acting as a reinforcing agent” (Hull, 1952 
P. 6); and they suggest that “the prior establishment of a stimulus as 
a diseriminative stimulus is sufficiently important to be included as a 


necessary condition in such general statements as Hull's” (p. 184)- 

lur a anei looked as if there were additional support for the 
Schoenfeld-Dinsmoor hypothesis in the fact that no onc. as yet, had 
been able to. impart clear-cut secondary reinforcing capacity to 2 
stimulus associated with shock termination (sce Chapter 6). Here 
be terminated without the subject's doing any- 


effective criterion, 
that had been associ 
Group I, but not for Gr 
lishment of a stimulus as a € 
for that stimulus to become 4 


vond Hull's 
; repeatedly 


As McGuigan & Cn 
ate of affairs 


postulate that a neutra 
and consistently in close 


the drive state could 
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animal was, for example, rewarded only after pressing the bar twice, we 
should not think of this as press-failure, press-reward, but rather as press- 
ress-reward (p. 184). » . i 
» When "response" is thus redefined in terms of the whole pattern. of 
behavior which proves effective in producing reward during acquisition, 
the Humphrey: paradox disappears. When “ esponse" is thus redefined, it 
is found that intermittent reinforcement, far from producing greater habit 
strength," actually produces reliably less than does continuous reinforce- 
ment (p. 198). 


As early as 1938 Skinner had advanced one version of this explana- 
tion; and in 1949 Virginia Sheffield (following suggestions derived 
from Hull) advanced another quite. similar 


onc. However, in. 1954 
Weinstock published an experiment. which 


put this hypothesis under 
considerable duress. The gist and genious of the Weinstock experi- 
ment was that he used widely spaced trials (24 hours apart) and still 
found that intermittent reinforcement led to 
which it could not readily do if the 
valid. This hypothesis presupposes that 
particular action are required to com 
soon enough that the stimulus 
the last repetition of the action 
occurs. If successive perform 
separated in time, this c 

By making certain 


the Humphreys effect, 
response-unit hypothesis were 
when several repetitions of a 
plete an act, the reward comes 
trace from the preceding as well as 
are still present when reinforcement 
ances of a given action are widely 
xplanation would not be applicable. 

assumptions about “memory” and "imagery" (see 
Mowrer, 1960, Chapter 5), it might still be possible to salvage the 
Tesponse-unit hypothesis as an overall explanation of the Humphreys 


effect; and even Weinstock does not exclude the possibility that it may 
partially account for the observed results under ; 


conditions of intermittent reinforcement 
findings (which have been subsequently: co 


the ordinary (massed) 
However, the Weinstock 


: nfirmed) strongly suggest 
Lie In any case, some other principle (or principles) is also operative 
herc. 


The discrimination hypothe 
termittent reinforcement le 
does continuous reinforcen 
cumstances the subject, 


sis, 


as it has been called, holds that in- 
ads to greater resistance to extinction than 
ient for the reason that in the former cir- 
having experienced runs of nonrcinforced 


cannot casily determine 
ended and "true" extinction 


inforcement this ch 
extinction th 
"loses. hope" 


trials during acquisition e i 
= l , when acquisition has 


has begun, w hereas with continuous re- 


ange is abrupt and clear-cut, In other words, in the 
at follows continuous reinforcement, the subject quickly 
and becomes frustrated and discouraged (sec. Chapter 
11); whereas in extinction following discontinuous xeu spei the 
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subject cannot be sure for a long time that there will be no more 
reinforcements and so “keeps hoping” and responding. In the one 
: -asilv and clearly distinguished from 
acquisition because the conditions are so obviously different; whereas, 
in the other case, acquisition and extinction are less markedly different 
and discrimination is correspondingly more difficult. This hypothesis 
has received widespread empirical support (Fehrer, 1956; Jones, 1953; 
Capaldi, 19575 and Lewis & Duncan, 


case, the extinetion procedure is 


Vvler, 1956; Parducci, 19575 
1956, 1957). 
While plausible enot 


the discrimination. hypothesis has 
"noncognitive") learning theorists; so. cf- 


forts have been made to find alternative interpretations of a less “men- 
thy of these is a study reported 


talistic" sort. One of the most notewor 

investigator's thinking went as follows. In 
reinforcement procedure, reinforcement is 
respect to primary reinforcement. Since, 
ced trials, the subject enters a compart- 
f stimulation which have previously 
reinforcement, then, here, too, secondary 


reinforcement will be experienced. Pur loosely, the idea would be 
that the subject has a sense of having arrived at the “right place,” even 
though on this particular “nothing there.” Hence it 
might be argued that in so-called intermittent reinforcement, rcin- 
forcement is really continuous; and because the total number of trials 
"ter here than in a control group receiving continuous 
intermittent primary reinforcement may well 
yent (primary and secondary combined) 
inforcement alone. If this were the 
to extinction would be in 


common-sense approach, 


igh from an intuitive, 
very acceptable to the 


not been 


More objectively minded (° 


by Denny in 1946. This 
à so-called. intermittent 
truly intermittent only with 
on the so-called. nonreinfor 
Ment or experiences other forms © 
been associated with primary 


trial there is 


Is likely to be greate 
primary reinforcement, 
result in more total reinforce 
than docs continuous primary 


Case, a correspondingly greater 


re 
resistance 


ho way enigmatic. 

The experimental evidenc 
ports this hypothesis but is it 
Instead, let us consider 3 simpler 
the same investigator in 1948- 1n an OF 


c cited by Denny in the 1946 paper sup- 
tricate and will not be described here. 
procedure which was reported by 
-dinary T-maze. the usual pro- 
cntical goal boxes at the cnds of the two "wings" 
th fhod in only onc of these. let us say the one 
s of such differential reinforcement the 
.a right turn at the choice point 
“reasoned that this kind of learning 
if the two goal boxes were them- 


cedure is to have id 


of a 
of the maze but WI ! 
on the right; and on the bas! 
Subject learns to make. consis 


rather than a left turn. But Denny 
could be considerably facilitated 
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i re d ee E 
lves clearly differentiated, so that there would be no (or at le 
selves arly 
less) generaliz 


tion of secondary reinforcement from the rew arded 

rr. . -— T ds 
to the unrewarded goal box. The obtained results supported this ir 
ference. Said Denny: 


These findings are interpreted as substantiating is paw Ma b 
principle of secondary reinforcement. It is assumed that in jue xod pe 
box group the s econdary reinforcement. gener; ized from Hur taena e 
box to the incorrect. end. box, providing reinforcement for the rat 
wrong responses; while in the unlike end-box g 
is presumed to be negligible and the animal i 
one side (pp. 248-249). 


roup stimulus generalization 
s consistently reinforced on 


This finding (which has been confirmed 


and extended by others. 
e.g, Ehrenfreund, 


1954), that secondary reinforcement. can generalize 
from one goal box to another (similar) one 
the assumption that it can 
one trial to another, 
underlay Denny’s 
in terms of reinfo 
sistance to extinct 


. makes entirely reasonable 
also generalize, in the same goal box, from 
This, quite simply, was the assumption W hieh 
1946 experiment and the attendant effort to explain. 


reement theory, the phenomenon of increased re- 


ion following intermittent 
However, even this inte 
unchallenged. In 1957 
marized as follows: 


primary reinforcement. 
"pretation, reasonable as it is, has not gone 
Mason reported an experiment which he sum- 


Fach of four groups of rats w 
gray and a white-gr 
the black or white 


as trained concurrently on both a black- 
àv discrimination problem [see Fig. 12-9]. When used. 
stimulus alley was positive and the grav alley was 
always negative. Fach group learned one of the di ainal 
partial (50 per cent) reinforcement of the 
the other discrimination under 
of the positive alley, 


and the other two g 


scriminations under 
positive alley, while they learned 
TUE complete (100 per cent) reinforcement 
Two groups received 300 trials on cach discrimination. 
TOUps received 200 trials on the completely reinforced 


Fig. 12-9. Floor plan of discrim- 
Gray ination apparatus used by Ma- 

i son (1957, p. 265) in testing the 
i ma 


Denny hypothesis. 
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discriminati 
nat i ri i inf 

ion and goo trials on the. partially reinforced. discrimination 
the animals were required to choose between 


Subse ; 
ibsequent to this training. 
of the initially learned discrimination (ie 


Vergine positiv e stimuli 
= and white). 

Fis id F. the prediction based on 
ieee eu pericu reinforcers in strengthening 
liem th E ia a and in agreement with a prediction derived 
aa] a irunswik (1935) theory of expectancies based upon envi- 
xime oi to a rilities, 17 of the 20 animals under each condition chose the 

alley which had been completely reinforced during the original 


trainir Thi T dui A E 
ig. This ditference Is significant beyond the .o1 level of confidence 


(p. 268). 


the Denny hypothesis concerning 
habits under con- 


Vo this unusually complete yet suecinet summary, it need only be 
"ree ro test trials, with both of the positive (black 
os ^ end boxes av able; and a choice of onc of che boxes on 

more of the 10 trials was regarded as constituting a preference.” 
the Mason experiment would seem to be 
on the Denny hy pothesis: if 
result. of summating with 
produces greater total rein- 
hat Mason's subjects should 
rather than con- 
1 


a ` 
dded that there were 


Fr i ae 
A rom one point of view, 
ery fair and straightforward 
reinforcement, aS 


mee d | attack 
nitent. primary a 
i; reinforcement. 

Id indeed appear € 
in which intermittent 
had been experienced. But there i 
by no means certain that preference 
; function. If, for example, 
one of which (when 
ad the other 


ne secondary 
ae then it wou 
ad cathe o the goal box 

Is primary reinforcement 
it is 
y test the same 
o levers, 
half the time ar 
trial, he would ung 
rs present. he found that the 
yf at all, he might very 
"empty ) and thus stop 
operating the 50%- 


iege oon iidiga here: 
al esistance to extincto 
OR being had a choice 
of Me singly) had previously is 
Nite had "paid off on every € 
Se the lame: Bur tf with both leve 

was not paying | 

out of order (oF 
would stop 


between OW 


“paid off" 
uestionably 


ri steintanenttiern lever 

Verri p that it was E 

aif mg it much sooner than he 
nreement. lever. 

thes another time (Mowrer, ne 

ee of “probabilistic eart Ts 
es a choice between tW? (or more 

atios, W 


Chapter 9) we shall discuss the 
“ and will then sce that it in- 
alternatives with 
fect, strictly 


r, 1960. 


response 
Humphreys € 
extinction situation, There- 
a combination of the two 
tation. We shall, there- 
ain an €x- 


differ ac t6 
oes reinforcement PF. chereas the 
Malina: 7 NES = AG! an 
f aking, is limited to ave respo ar 

re : ^d 
€ the Mason experiment js sol 


cub : " p " 
risk correspondingly difficult of 
n turn to yet anothe 


respect to this di 


nse in 
nething of 
interpre 
r attempt to obt 


fore ; 

re E 

>in the ne "ctio "b 2 
jest sce Jemma of intermittent re- 


Perime Be : 
infi mental decision. with 
Orcement. 
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VII. The Bitterman-Feddersen-Tyler Experiment 


In 1953, Bitterman, Feddersen, & Tyler published a paper entitled 
da < . H $ tig. is i 
“Secondary Reinforcement and the Discrimination Hypothesis, which 
they introduced as follows: 


The experiment to be reported is concerned with the relation che ie 
the concept of secondary reinforcement and what Mow rer anc i 
have called the discrimination bypotbesis—the assumption that rate of 
extinction is inversely related to the simil: rity. between. conditions of 
training. and extinction. In contemporary learning theory, the effect of 
change in the afferent consequences of a response upon its resistance to 
extinction has been considered primarily in terms of i 
secondary reinforcement—a neutral stimulus acquires reinforcing properties 
as a function of repeated association with primary reinforcement (Hull, 
1943). From this point of view the extinction of a response must inevitably 
be retarded by stimuli which have been contiguous with reinforcement 
during training. It is doubtful, however, that the effect of change in the 
afferent consequences of response can be fully understood in this wav. 
The presence during extinction of stimuli encountered in training may 
contribute to the similarity of the two series of events and. thus, to some 
extent at least, sustain response independently of previous association with 
reinforcement. The validity of this conception may be tested. under 
circumstances in which the transition from training to extinction is more 
readily discriminated in the presence of a secondarily reinforcing stimulus 
than in its absence (p. 456). : x 


the principles of 


Before examining the results of this experiment, let us review the 
procedure followed and the logic thereof, 
ploving the animals which the 
50%¢ intermittent reinforceme 
with a white goal box on reinforced trials and a 
nonreinforced trials (or vice versa 


The main experiment, em- 
authors refer to as Group H, involved 
nt for running a straightaway alley, 
i black one on the 

> AS 3 counterbalance for the pos- 
sible effects of color per se) (cf. Denny, 1948; Saltzman, 1949; also 
Chapter 4). In order to prevent the subjects from se zoal 
box was present from the start of a run ( 
running on the “negative” 


cing which g 
and thus perhaps just not 
rimenters introduced a gap 
subjects had to leap across. Im- 
a card (with black and white 
as an animal jumped 
to the goal box but preventing 
were kept from secing what the 
actually in it. 


trials), the expe 


in the maze, near the end, which the 


mediately beyond was vertical stripes on 
it) which fell over against it, thus giving access 
into it. In. this wav subjects 
goal box looked like until they were 


Vision 


After 10 days of training (10 trials per day) with the procedure 
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just described, the rats were subjected to extinction; but they were 
now divided into two subgroups, Group H-N, which on all ex- 
goal box that had been the nonreinforced 


tinction trials found the 
iroup 1-8 w hich consistently. found the 


onc during acquisition, and € 
ad been reinforced during acquisition. In other words, 
ox was white on rewarded trials and 
then Group H-N had only the black 
iroup H-S had only the white goal 


goal box that h 
if during training the goal b 
black on nonrewarded trials, 
goal box ¢ uring extinction and € 


box. The results are shown in Fig. 12-10. 
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man, Feddersen, & Tyler (1955) show- 
to extinction (faster. running time) of rats (Group 
the nonreinforeed goal box during extinction. as 
ncountered the previously reinforced goal 


Fig. 12-10. Curves reproduced from Bitter 
resistance 
1 only 
II-S) which e 


ing the greater 

H-N) which encounteree 

opposed to rats (Group 

box. 

the animals which, during extinction, encountered 
nonrcinforced goal box showed much greater 

int) resistance to extinction than did the animals 


tion encountered only the previously reinforced 
firmation of the "discrimination hypothesis": 


extinction. encountered conditions 


As will be scen. 
only the previously 


(statistically signifi 
which, during extinc 
one. Here is dramatic con t 
the Group TIEN animals during 


(nonreward in bl goal box) which had also existed (though not 


acquisition and so, not being able to separate 
“hoping” and responding; whereas the 


ack g 
so protractedly ) during 


"iur by SESS 
acquisition and extinction. kep 
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Group H-S animals during extinction encountered dep Hes 
reward in white goal box) which had never existed before, anc so P 
mediately reacted to extinction as extinction, rather than as continuec 
ac sition. A í 

or this sword cuts both ways: while affording striking confirmation 
of the discrimination hypothesis, the results of the Istum: 
Feddersen-T vler study at the same time challenge a hypothesis w hich 
is far more basic to our present analysis: the notion that "habit" is the 
secondary reinforcement which an organism experiences as a result 
of performing (or “considering’—see Mowrer, 1960. Chapter uS 
given responsc. Here a response that brought the subject into. the 


white goal box, with its very considerable S* potency, extinguished 
sooner than did the same response when it brought the subject ure 
the goal box which presumably had Jess St capacity. And the authors, 
taking this observation at its face value, conclude that: “This outcome 
cannot be understood in terms of the concept of secondary reinforce- 
ment” (p. 461; see also the passage already quoted from p. 456). This 
position, it scems, is based on an incon 


iplete analysis of the results 
of this 


investigation. While the results cited do indeed. support the 
discrimination hypothesis, at a gross, molar level of analysis, they do 
not actually impugn either the secondary-reinforcement conception 
of habit in general or, for that 
forth by Denny, as will now be 


Thus far only half of the subjects used in the Bitterman-Feddersen- 
Tyler experiment have been described, i.e., the Group-II. (S and N) 
animals. The Group-I animals. (not hitherto. mentioned) were also 
given 50% reinforcement during acquisition; but instead of running 
into the white box on reinforcement trials and into the black box on 
nonreinforced trials, they ran into the same (white) box on rein- 
forced and nonreinforced trials alike, 
mittent-reinforcement experiment. Then, for the extinction test, the 
Group-I animals were divided into two subgroups, one of which 
(I-S) continued to find the same (w hite) goal box at the end of thc 
allev, whereas the other (I-N) found a differently colored. (black) 


goal box. The extinction results for the I-S and I-N animals are shown 
in Fig. 


matter, the particular hypothesis set 
shown, 


as in the usual type of inter- 


12-11. Taken in conjunction With the results shown in Fig. 
12-10, these findings have a number 


of implications: 

1. First we note that the curves in 
factorily to the notion that habit 
we see that those subjects which 


Fig. 12-11 conform quite satis- 
is secondary. reinforcement, Here 
ran into the same goal box during 


extinction as the onc used during acquisition (I-S) showed visibly 
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resistance to extinction (habit strength) than 


(and reliably) € 
did tl SIS m - 
vose which ran into a goal that had never been associated with 


food ; 1 "A. "Ti 

Aie and hunger reduction (I-N). This outcome i5 strictly in accord 
1 ^ x . B H . i 

a the hypothesis that habit strength is progressively lessened as 
isual stimulus consequences of a 


rreater 


ene ves 
E eliminates or changes the v 
abitual” AR (ait 

ibitual ( positiv clv reinforced) response. 
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d E 
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ive to put the I-S curve and the II-S 
as shown in Fig. 12-12. The I-S 
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— between the two curves shown in Fig. J212 is therefore the 
one typically obtained in conventional experiments on intermittent 
versus continuous reinforcement. The I-S animals had difficulty in 
discovering where acquisition ended and extinction started, whereas 
the transition was perfectly obvious for the [I.S animals; hence the 
much more rapid extinetion (as the discrimination hypothesis would 
require) of the latter, 
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Fig. 12-12. Curve II-S. (from Fig. | 
produced on the same 


kee ving Wit , + " * "ec 
ping with that usually È tuntinuoüs and intermittent reinforcc- 
ment, 


2-10) and curve LS (from Vig. 12-11). re- 


Iparison, “The Observed difference is in 


3- It is also instructi 


ve to superimpose the I-N and the II-N curves 
upon the same coordi 


nates (scc Fig. 12-13) 
25 of the points in the J-N curve 
Whereas only 


- Here it turns our that 
are above the 4.3 latency lever, 
cite !5 of the points in the H-N curve are similarly located. 
This means, simply, that the l-N. animals extinguished more easily 
than did the [I.N animals. Here again, during earned both groups 
were treated alike objectively, Le ran into la same (black) extinc- 
tion box. But this box had a different history ("meaning") for these 
two groups. The J-N animals had never before zi 
the H-N animals had encounte 
never received food in it. 


encountered it, whereas 
red this box 


during acquisition but had 
How ever, 


the fact that it sustained the 
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running habit during extinction better in Group I-N than it did in 
Group I-N suggests that the black ("nonreinforcement") box in the 
H-N group had nevertheless acquired a good deal of secondary rein- 
not, presumably, have come about 


forcement value. This could 
reinforcement (sec Denny and 


through generalization of secondary 
Ehrenfreund), since the two (reinforcement and nonreinforcement) 
goal boxes used in Group-lI training were so manifestly different 


(white versus black). But there was another mechanism w ‘hereby the 


black goal box could have acquired SF potency. Since the interval be- 
tween trials in this experiment was only 20 seconds, an animal might 
easily run into the black (empty) box and, well before the stimulus 
trace of this experience had disappeared, it might find itself in the 
white (baited) box; and since Jenkins (Chapter 10) has shown that a 
stimulus can acquire secondary reinforcement. with temporally de- 
lay ed primary reinforcement, there would be no reason to deny such 
a possibility in the present study. However, Elam, Tyler, & Bitterman 
(1954) have since repeated the Bitterman-Feddersen- Tyler study with 
distributed practice (a minimum of 15 minutes between trials) and 


have obtained much the same results, which the authors interpreted 
imination hypothesis” and dis- 


as lending further support to the “dis 


15 a a Oe Se i i 
[] 
1.4}- 
1.3} = 
1.2 }- 
È 
5 11 Al 
E 
z10|- J 
= e LN 
0.9 
P. YAR, B | II-N 
os, hy 
; à ae 
07 - u L Lit 1 i LT 
91 1001 10 20 30 
Trials 
Fig. 12-11) re- 


` ^ : "i 2-10) and curve I-N (from 

Fig. 12-13. Curve H-N (from Fig. 12-10) à d ury 1 Fig 

produced on the same coordinates. The superiority of the H-N performance 
end box into which rats run on the nonreward trials in 


Shows that a. distinctive OX | x 
intermittently reinforced acquisition acquires some secondary-reward value. 
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continuous reinforcement), it becomes a cue for reward; and when 
hope becomes thus conditioned to the experience of nonreward, this 
experience is decidedly less frustrating, less disappointing, less anger- 
producing than when there is “no hope. a 

Another way to put the same notion is to say that intermittent 
reinforcement of a response does not make the response actually more 
resistant to extinction; instead, intermittent reinforcement makes ex- 
tinction less effective, in the sense that the intermittent reinforcement 
during acquisition neutra 


s the normally active agent in extinction, 
namely frustration (anger, disappointment). Asa result, the subject 
remains hopeful—and  "habitful'—longer (sometimes remarkably 
longer; see Zimmerman, 19: 1959) than would otherwise be pos- 
sible (cf. Kendler & Lachman, 1958, on intermittent. reinforcement 
and resistance to habit reversal). 

"Thus, after a good deal of confusion and "lost motion," we scem 
to have arrived at a thoroughly satisfactory explanation of the 
Humphreys paradox. Our error has been in 
persistence. of a 


assuming that the greater 


habit following intermittent reinforcement. neccs- 
sarily implied greater "habit strength," 


greater persistence occurs because the 


has, under these circumstances, been |; 
si: 


instead of realizing that. the 
normal machinery of extinction 


argely put out of. operation. Re- 
ance to extinction has been regarded as one of the most reliable 


means of assay ing habit strength, Now we see that, at least under some 
circumstance, it may be extremely: untrustworthy. If, by the means 
described, the normal effectiveness of nonreward iensiention) is 
lessened, then, obviously, a habit may show heightened resistance to 
extinction, not because the habit itself is stronger, but because the 
forces making for extinction have been veeabeneq. i 


There is only one "catch" in the foregoing argument: How, it 
may be asked, can nonreward on one trial 
stimulus for hope if the next (rewarded) trial is delayed bevond the 
normal span of the gradient of reinforcement (w lich, as we have seen 
in Chapter 10, depends upon the phenomenon of simus tracc)? 
There is apparently only one satisfactory: w ay of answering this ques- 
tion, and that carries us bevond the scope of the present ind (see 
Mowrer, 1960, Chapter 5). However, this ipee rests upon a well- 


authenticated set of observations, and we need have no further con- 
cern about the problem for the present. 


The plausibility and intelligibility: 
perhaps be increased by employing tl 
text of active punishment, rather th: 
in Chapter 11, nonreward 


become the conditioned 


of the foregoing argument can 
ie same assumptions in the con- 
an mere nonreward. If, as assumed 


is functionally: equivalent to mild punish- 
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ment, and if continuous nonreward (as in extinction) is less inhibitory 
when, during acquisition, nonrewarded trials have been interspersed 
with rewarded ones, then it would follow that the effect of punish- 
ment would be similarly lessened if occasional punishment were ex- 
ries of rewarded trials, rather than being applied 
—from some given point onward. 


perienced during a se 
consistently—and for the first timc 
f that one group of rats were allowed to press a 
receiving a pellet of food cach time, and that 
punishment were then consistently applied on each subsequent trial, 
in the form of an electrical charge on the bar. There can be little 
doubt that this would be an effective way of quickly inhibiting the 
abit. On the other hand, suppose that, during acquisi- 
only now and then and that 


Suppose, for example. 
Skinner bar 100 times, 


bar-pressing h 
tion, the charge 
the animals, after a pun 
charged. Here punishm 
ment, but by no punishn 
casily anticipated: when, 


were put on the bar 
ished bar press, would find the bar no longer 


ent would be followed, not by more punish- 


and continued reward. The result is 
the punishment was con- 


rent— 
after 100 trials, 
sistently applied, it would probably be far less inhibiting than in the 
other group of animals. 
We might say that these animals, by the procedure described, had 
developed “courage” (Chapter 11); or, equally, we might say that 
ver rigid, foolhardy. If we, the observers, wished to 
r-ride and ignore the shock, we would evaluate 
v—and call it couragcous; but if we wished 


ats persisted in it, we would then evaluate 
same principles, it seems, would 


see the subjects ove 
their behavior positivel 
to block the habit and the r 
their. behavior negatively. But the 
be involved in both cases. 

No onc. apparently. has investigate 
posed. Hilgard (1956) reviews the somewhat equivocal studies by 


Estes (1944) under the following headings: “Punisnment Docs Not Act 
as a Negative Reinforcement” (p- 110) and Intermittent Punishment 
Is Afore Effective ‘Than Punishment at ery Occurrence” (p. 111). 
most relevant finding of the Estes research is that: 


1 this problem precisely as pro- 


Perhaps the 
ment is delivered. every time the response is made, the rate of 
is seriously depressed. While it is not depressed as much if punish- 
m g ffects of punishment persist longer in the 


only occasionally; the e 
Hilgard, 1956; P- 110). [See also Akhtar, 1960. ] 
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of the most interesting topics in the field of learning pois 1 i duin 
becomes quite important w hen we consider its implications i the ap em 
ship between child-rearing practices and personality dyi ades oe e wa 
ment. Sears and his co-workers (1953), on the basis of their rence 1 s : 
child-rearing antecedents of aggres: on and dependency: in arg £ dp 
theoretical hypothesis that the combination of positive reinforcement us 
moderate frustration or punishment establishes a drive | habit | of D 
strength. Too much or too little frustration results in a weaker drive. < ist 
Shaffer and Shoben (1956) suggest that frustration generally strengthens a 
drive. "pu 

The present investigation is basically related. to the above bine 
and ideas, and mainly examines the effects of positive reinforcement with 
shock punishment in terms of resistance to extinction using albino rats as 
subjects in an instrumental conditioning situation (p. 1). 


In experimentation with rats, Tsushima was able to show that inter- 
polation of occasional punishment during the acquisition of a habit 
produces greater resistance to extinction (consistent nonreward); but 
he did not explore the problem of whether greater resistance to pes 
ishment can be similarly induced. Superficially there is a contradic- 
tion between the Estes finding that punishment has a more persistent 
effect if intermittently applied and HM 
tent punishment is less effective than consistent punishment. This para- 
dox probably hinges upon certain quantitative and other variables 
which need not be considered here,* 


Tsushima's finding that intermit- 


"The. writer has recently. had 
the Humphreys effect. with P 
interesting and instructive que 
plicable where the habit is 
ment might well be 
the fact that 


Occasion to discuss the Amsel interpretation 2 
rofessor Humphreys himself, who raised a very 
stion. Granted that. the Amsel hypothesis is ap- 
a positive, goal-oriented one (and where nonrcinforcc- 
expected to generate frustration), could one similarly explain 


a defensive response, such as the 
greater resistance to extinctioi 


conditioned eyeblink, also shows 
n following intermittent, rather than continuous, 
reinforcement? Here the omission. of reinforcement would be expected to. gen- 
erate relief rather than frustration, so that the Amsel hypothesis might not seem 
to apply. On the grounds that evelid conditioning is Something of an anomaly 
and probably does not represent conditioning in its most characteristic and sig- 
nificant form, let us shift the example from eyelid conditioning to fear condi- 
tioning, as indexed by the psychogalvanic reflex. Here, too, the Humphreys 
effect is known to hold; and the Amsel hypothesis applies quite nicely 

The Amsel explanation of the extinction. of a hunger- 
if the habit is executed with the expectation of res ard 
and none is forthcoming, then anger supervenes and counterconditions the hope 


(tvpe-2 secondary reinforcement) w hich presumably constitutes. the habit 
“strength.” And as was also pointed. out in Chapter ii 


been acquired and is to be extinguished is fë 
stimulus is presented and not confirmed, 
counterconditions the fear, Now wh 


as onc 


may sav. in reverse, 
motivated habit is thar 


Jif the response which has 
ar, then cach time the conditioned 
a sense of relief is experienced, which 
at presumably happens where a PGR has been 
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IX. The Discrimination and Counterconditioning 
Hypothesis Inseparable 


Sections VI and VII suggest that, of many possible explanations of 
the Humphreys paradox which have been advanced, the discrimination 
hypothesis is in many ways the best. And in Section VIII, it is shown 
how the discrimination hypothesis can be given a noncognitive refor- 
mulation, along the lines recently suggested by Amsel to explain cx- 
tinction, in general, as counterconditioning. But the essential identity, 
or at least equivalence, of these two ways of interpreting the Hum- 
phreys p: radox becomes clearer still if we examine two experiments 
which have been reported by Tyler, Wortz, & Bitterman (1953) and by 
Capaldi (1958). ‘These experiments have already been briefly alluded 
ial consideration for the reasons just given. 

In the first of these experiments, two groups of rats were trained to 
raightaway alley for food under the following condi- 
forced during acquisition go% of the 
t patterns. In the Group-A animals food 

y trial, ic., alternately; whereas in the 
Group-R animals food was given also ṣo% of the time but randomly. 
When tested with no reinforcement. whatever, the Group-A animals 
dedly faster than did the Group-R animals. 


to but deserve spec 


negotiate a st 
Both groups were rein 
but according to differen 
rly every-other 


tions. 
time, 
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is of intermittent reinforcement is that when, during acquisition 
on a given trial, is not followed by 
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set up on the ba 


training, the danger signal is presented and, 


shock but on the next trial is so followed, nonreinforcement loses its cap: 
is rendered. less effective in counterconditioning the fear 
phat is to say, when nonreinforcement of a 


bring relief, and so 
luring extinction. proper. 
and the resulting relief on certain trials during acquisition training 
continued reinforcement, but again by reinforcement, then 
nforcement (sce Section IV) is structured in such a 
f-producing value of nonreinforcement, when it be- 
od, just as the recurrent withhold- 


reaction € 
danger signal 
is followed, not by 
e sign function of nonrei 
as to reduce the relic 
uous in the true extinction. pe 
reinforcement during acquisition training can structure the sign 
sity and thus render 


th 
way 
comes contin 
ing of positive i 
function of such an event so as to reduce its frustrating capa 
continuous nonreinforcement less effective during extinction. proper. It is pre- 
at this tv pe of explanation is also. applicable, with perhaps minor modifi- 
Humphreys effect as observed in eyelid conditioning and other 
al responses. It is, incidentally, also interesting to ask w hether 

xtinction (see Chapter 11) can be similarly used to resolve the 


the fatigue theory of e apte in be rly us 
Humphreys p? jade, The preacnt writers tentative impression is that it cannot. 


And if this impression should be sustained by further research and analysis, this 
yet another indication of the greater potency of the frustration theory 


sumed th 
cations, to the 
latency skelet 


short- 


would be 


of extinction, in general. 
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Now the cognitive version of the discrimination hypothesis would 
hold that the reason for this outcome is that the Group-R animals never 
“knew” when reinforcement was going to come during acquisition so 
could not be sure when truc extinction had started, whereas the 
Group-A animals did know when to expect reinforcement and could 
therefore detect quite precisely the onset of extinction. “But.” one might 
ask, “how can you possibly check such a hypothesis? How can vou tell 
what a rat does or doesn't ‘know’ or 'expect'?" In answer to this ques- 
tion, Tyler, Wortz, & Bitterman published a graph which is re- 
produced here as Fig. 12-14. Here the average running times of thc 
Group-A animals on cach of the 120 acquisition trials (10 trials per 
day for 12 days) are clearly plotted, with the open circles representing 
nonrewarded runs and the closed circles, rewarded runs. During roughly 
the first half of the acquisition trials, there is no systematic difference 
in running speeds on the alternately rewarded and nonrewarded trials. 
But in the second half of the acquisition trials, it is unmistakable that 
the rats “knew” what they were doing, or at least what was being 
done to them. On the nonrewarded trials they ran regularly and 


markedly slower than on the rewarded trials, thus indicating that they 
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Fig. 12-14. Graph reproduced from Ivler, Wortz 
the gradual discov ery (by rats) of an alterr 
for runs in a simple alley-type maze. 


& Bitterman (1953) showing 
aung pattern of reward and nonreward 


A “Knowledge” of this pattern significantly 
influences the subjects! behavior when rew ard is withdrawn entirely (sec test). 
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had learned the alternating pattern of reinforcement and nonreinforce- 
ment and were decidedly less “hopeful” and less hurried on the non- 
ded trials than they were on the rewarded ones. Hence, one can 


rewar 
say with confidence that the Group-A rats would know, quite precisely, 
when extinction had started, i.e., when conditions had been changed to 
no reward at all, and would modify their behavior accordingly; whereas 
the Group-R animals could not make nearly so sharp a discrimination 
and would be correspondingly slower in modifying their behavior fol- 
lowing the withdrawal of reward. 

"The experiment reported by Capaldi shows the high pertinence of 
the discrimination hypothesis in an even more dramatic form. Using 
essentially the same type of apparatus as was employed in the study 
just described, this investigator ran four groups of hungry rats under 
the following conditions: Group A-14. 14 acquisition trials with al- 
ternate reinforcement and nonreinforcement; Group A-7, seven ac- 
alternate reinforcement and nonreinforcement; 
reinforcement; and Group R-7, 


quisition trials with 
Group R-14 14 trials with random 50% 


seven acquisition trials with random 50% reinforcement. Capaldi gives 


the rationale of his experiment thus: 


ation hypothesis holds that the ease of discrimination be- 
tween extinction and acquisition trials is a factor in extinction. If increased 
amounts of training result in greater discriminability between acquisition 
and extinction trials, then resistance to extinction. should decrease with 
overtraining. Alternating partial reinforcement may fulfill the conditions re- 
quired by the discrimination hypothesis. Incr ed amounts of training under 
an alternate reinforcement schedule may lead to better learning of the 
pattern of reinforcement itself because of the predictability of the pattern, 
i.c., reinforcement follows nonreinforcement and vice versa. Better learning 
of the pattern implies that deviations from the pattern. will become more 
and therefore the extinction. process should be accelerated. Ex- 
tinction following random reinforcement should not be especially acceler- 
ated by increased amounts of training because of the unpredictability of 
the pattern and therefore the impossibility of learning the pattern (pp. 


367-368). 


The discrimin 


noticeable, 


s, using the discrimination hypothesis, Capaldi made 


In other word 
‘© alternate reinforcement 


arkable prediction that if regular 5c 
acquisition, animals which received ore training 


the rem 
were used during 
would have a better grasp of the reinforcement-nonreinforcement pat- 


would therefore discriminate more precisely when extinction 
nonreinforcement) had set in, and would thus show less 
This prediction is strikingly confirmed by 
12-15. Here we sec that although the 


tern, 
(complete 
resistance to extinction. 
the findings shown in Fig. 
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Group A-14 animals had received twice as many acquisition trials as 
the Group A-7 animals, they showed decidedly less resistance to Cx- 
tinction, i.e.. reacted more promptly and more appropriately to the 
change in reinforcement pattern which marked the onset of extinction, 
Moreover, the Group R-14 and R-7 animals provided perfect controls 
for this interpretation; for here we see that when there was nothing that 
could be learned about the pattern of reinforeement-nonreinforcement 
(because it was random), additional training did vor result in reduced 
resistance to extinction. li 


Here, and in the Tyler-Wortz-Bitterman experiment, is about as 


conclusive support for the discrimination hypothesis (as over inst 
most other interpretations) as one could ask. However, these findings 
are equally interpretable in terms of the frustration or countercon- 
ditioning hypothesis, In. both experiments it is shown that if a rat 
expects reinforcement on a given trial and it is not forthcoming, this 
failure of reinforcement has a markedly depressing effect upon the rat's 
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: Tt G traning in a straightaway alley (Group A 14. 35 
compare Ss oup A D with alternate reward and | display Jess 
resistance to extinction (as indicated by slower running). This difference does not 
appear in comparable groups (Roig e 


partern of training (random reward 


and R->) which have undergone another 
j ee ‘ and reinforcement), These findings strongly 
support the discrimination. (and frustration) interpretation of the Humphreys 
effeer 
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"morale"; whereas, if a rat has no ce tain knowledge or strong expecta- 
tion of when reinforcement will or will not occur, failure or with- 
drawal of reinforcement has a much less decided effect. As already 
noted, these findings conform exactly to predictions generated by the 
discrimination hypothesis, but they are equally congruent with the 
frustration hypothesis. If a rat is vot sure whether a given trial is the 
one which is to result in reinforcement, he is not particularly frustrated 
disappointed) if there is no reinforcement; but if. on a par- 
reinforcement iy confidently expected and does not ma- 
aximum frustration and correspondingly 
of hope and habit strength. 


(surprised, 
ticular trial, 
terialize, then there will be m 
counterconditioning (extinction) 
it seems that the discrimination hy pothesis 
hy pothesis are, in reality, one and the 
at the one is formulated in “cognitive” 


rreater 

As already indicated, 
and the counterconditioning 
same, the only difference being th 
other in affective-dynamic terms. In this book we have given 
affective rather than cognitive learning; but the 
ibed (along with many other considerations) sug- 
art 


n 


and the 
major attention to 


experiments just de 
affective event normally has its cognitive counter 


gv of learning must take the latter into 
However, the detailed demonstration 


gest that every 
and that a complete psycholo 


t no less than the former. 


accoun 
a task for the book which is to follow and cannot be 


of this thesis is 


undertaken in this volume. 


on Exemplified: 


X. Temporal Discriminati 
ve”) Behavior without 


Avoidance ("Compvlsi 
Objective Warning 
a revolvable cage and shocked every 20 seconds 


If a rat is put into 
e (by some minimal 


during this interval, 
an inch or two), à Very 


unless, it revolves the cag s A 
interesting type of performance 


amount—say i < 
ise cach "spontaneous (or interval) response, of w alk- 


results. Be 
ing or running, serves to avoid (or 
the neyt “trial,” it is clearly to the rat’s advantage to make these inter- 
val responses; and if they are made with sufficient reliability, the ex- 
of shock can be completely eliminated. Figure 12-16 il- 
that commonly occurs in a situation of this 
as a result of continuing training of this 


at least postpone for 20 seconds) 


perience j 
lustrates the learning 
And Sidman (1956). 


kind. 
kind much further, reports that rats and other laboratory animals 


become highly profic 
where shock is almost never allowed to occur (cf. 


eventually ient in such behavior, reaching the 
point eventually 
F 12-18, p. 4565 also sce Fig. 12-17. p- 479)- 
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From an intuitive and common-sense point of view, this finding is 
not very surprising. Yet, in terms of systematic behavior theory, it has 
oceasioned some puzzlement. As we have seen in earlier chapters 
avoidance behavior is usually instigated and reinforced, respectively, 


by the appearance and disappearance of some specific danger signal; 
Whereas, in the situation under discussion, no such signal is immediately 
apparent and, as a result, some writers have been inclined to see here 
a possible exception to the principles now generally regarded as gov- 
erning avoidance learning. However, a little closer scrutiny of this 
situation indicates that these principles apply here, too, very satis- 
factorily. 

Schaefer (1959) has pointed out that in a situation of the kind de- 
ibed, what first happens is this: the subject learns to discriminate be- 
ag in motion and the state of being at rest. Shock 
is never experienced by the animal when moving or just after having 
moved; whereas shock does occur when the animal is at rest. Therc- 
fore, the subject W ill soon begin to feel afraid when at rest and re- 
d safe when in motion—and immediately thereafter. Hence. 


se 
tween the state of beir 


lieved an 
the “danger signal” 
can terminate it by moving 
will be reinforeed by fear reduction quite as much, presumably, as it 
i signal were some external stimulus such as a 


is the state of rest or inaction; and since the animal 
(and revolving the cage), this behavior 


would be if the danger 


buzzer or a blinking light, which was turned off cach time the animal 
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ran. Hence we see how it is that the subject in this experimental 
setting would be motivated to engage ina very considerable amount 
of “interval” activity or responding: because this is the w av to elimi- 
nate the "danger signal." i.c., change a dangerous situation into a safe 
onc. 

However. on both theoretical and empirical grounds, we have reason 
to believe that a still more refined and efficient discrimination, or 
“adjustment,” can be made here. Strictly speaking, it is not true that 
action is safe and inaction dangerous, Action is, to be sure, safe enough; 
but inaction is not necessarily dangerous. It is dangerous only when 
upw ards of 20 seconds has clapsed since the preceding response. Therc- 
fore, instead of running, stopping, and running again almost im- 
mediately (as an inexperienced subject for a time tends to do), an 
animal can handle this situation—very nicely—by running stopping. 


waiting a time, and then, at the appropriate point, running again (sec 


ki 25 

2 

2 20 = 

8 Fig. 12-18. Temporal distri- 
& 15 bution of — "spontaneous 
$2 avoidance responses in a 
$ 10 well-trained rat (Sidman, 
g Ld T 
g 1956, p. 293). The response 
TR shock interval was again 20 
a s r1 £ 

= seconds, 

oe 

È m eS 


10 20 30 
Interresponse time in seconds 


Fig. 12-18). Here one thinks immediately of the mechanism of stimulus 
trace described in Chapter i 


i 10. For the improved type of adjustment just 
described. all the Š 


| the subject has to do is to learn to discriminate between 
the safe (earlier) and the really dangerous 


(later) parts of the trace; 
the shocks can thus still be 


avoided, and without a lot of unneces- 
sarv, useless responses in the meantime. 


In a well-trained, experienced rat one 


: can see this type of adjust- 
ment working very clearly, y 


: After having just made a run (or per- 
formed whatever other type of response 


; 2 is required in the situation). 
the animal will 


. appear quite unconcerned and relaxed for a time. 
grooming itself, rummaging about, 


) or doing whatever else strikes its 
ancy. But then, as if an inner "timer" 


i Of some sort were operating. 
the rat will presently. be "reminded" 


of the Impending danger and 


will again make the appropriate response. Because the stimulus trace 
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is of limited duration, there is a corresponding limit to the duration 
| that can be effectively used here; but within 


of the intertrial inter 
this limit, temporal discrimination (or conditioning), though diffic 
can nevertheless develop and function quite Ot TERE 
This phenomenon has rather obvious relevance to the psychology 
of "work" and to certain types of “compulsions” as they appear 
clinically; and Brady (1958b; also Brady, Porter, Conrad, & Mason, 
1958) has shown that monkeys which are subjected to certain sched- 
avoidance training of this kind develop lethal stomach ulcers. 
Mowrer & Keehn (1958) gives a general review of 
and Sidman and co-workers are vigorously, and 


ules of 
The article by 
this phenomenon; 
imaginatively, exploring its ramified implications. 


XI. Generalization and Apparent Failure 

of Extinction 
has been made throughout this volume (sec cs- 
rning, without occasional. reinforce- 
Hopes, without ful- 


The assumption 
pecially Chapter 11) that all lea 
ment, undergoes extinction. or 
id then, grow dim; and fears, if not periodically 


"oblivesenc: 


fillment at least now 
confirmed, subside. 
times been reported. 
tionally: autonomous" ( 


closer examination, usuall 
and even "memories," in the most subjective 


Sceming exceptions to this assumption have some- 
Instances of habits w hich appear to be "func- 
a term popularized by Allport, 1937), upon 
y turn out to involve subtle sources of sec- 
ondary reinforcement, 
1 unless kept alive by occasional recall. 


sense of the term, fade 
have a good review of some of the 


Hilgard & Marquis (1940) 
studies showing unusu 
been most extensively studied by Solomon and as- 
The apparatus used in this line of 


earlier al persistence of fear reactions; but the 
problem has 
with dogs as subjects. 


sociates, 
o, 12-19 and is described. by Solomon & 


investigation is shown in Fig. 
Wynne (1953) thus: 


box consisted of two compartments se ted by a barrier and 
a guillotine-type gate. he inside dimensions of cach compartment were 45 
in. long, 24 in. V ide, and 40 in. high. The barrier | which the subject had to 
leap over in order to go from one compartment to the other] was adjustable 
in height. The gate could be raised by means of a pulley system above the 


level of the top of the comp 
top of the barrier, thus creatin 
compartments. + + + 

"phe barrier was 
Therefore, the dogs coule 
the other when the gate was raised (pp. 2-3)- 


‘The shuttle 


artments. When the gate was closed it rested on 
g a solid wall completely separating the two 
adjusted to approximately the height of the dog's back 
1 look over the barrier from one compartment to 
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It should perhaps be added that the “danger signal” employed in 
the foregoing research involved two distinct. stimuli: 


The lights above the compartment in which the dog was confined were 
turned off [by the experimenter] by means of a foot pedal. Simultaneously, 
the experimenter raised the gate by pulling the counterweight on the 
pulley system. Therefore, the subject was in relative darkness but was, 
at the same time, exposed to the presence of the barrier, and immediately 
beyond it the other compartment was still illuminated. This situation was 
referred to as the conditioned stimulus pattern (CS), or the "signal" (p. 3)- 


The authors then summarize the procedure followed and the re- 
sults obtained in these words: 


An experiment in traumatic avoidance learning is reported in which 
dogs were trained to avoid a just-subtetanizing shock by responding to 
a signal which preceded the shock by a period of 10 seconds. A shuttle-box 
jumping response was reinforced as the instrumental avoidance reaction. 
The dogs received ten trials per day with a 3-min. interval between trials. 

The results can be stated briefly: (1) During the escape phase of learning, 


r Shuttle-box 
for dogs 


Fig. 12-19. Drawing of apparatus (reproduced from Solomon and Wynne, 1953) 
designed and used for avoidance learning in dogs. g 


GENERALIZATION, DISCRIMINATION, AND SKILL 483 


s hen the animals were recciving shocks because they were not “antic- 
ipating," the latency of the jumping response decreased rapidly and did 
not change significantly after the third trial. (2) The first avoidance re- 
sponse, a response to the signal having a latency of less than 10 seconds, 
appeared. (on the average) on the fitth trial; it was preceded at about 
the third trial by the appearance of conditioned emotional reaction to the 
) The latency of the first avoidance response represented an 
ase from the escape reaction latencies preceding it. (4) After 

the first avoidance trial, the larency of responses continued to decrease in 
manner, while the latencies of the escape reactions did not 
of avoidance reactions continued to decrease 
ast shock was received. (6) The difference 
ock trial and the avoidance trial following 
nitude of the CS-US interval. 

] avoidance reactions. showed 
100% escapes to 100% 
f shock received 
r of reversals 


signal. (3 
abrupt decre: 


an orderly 
change. (5) The latency 
after the trial on which the | 
m. latency between the last sh 
it was gr seconds, approximately the mag 
(7) ‘The sequential patterns of escape and 
phe some dogs make a complete change from 
Woidances without reversal. (8) Relatively large amounts 0 
during the first few trials tended to decrease the numbe reversa 
on the trials following the first avoidance response. 

as noted in conjunction with decrease In 


ee avoidance to escape 
a iri. ping of behavior W 
Tvs a reactions. EL D 2d 
of d sults were discussed within the Sup suce 1 
in Ni o Several inadequacies in current earning 
Ming to explain our findings (PP: 18-19): 


theory 


of a two-process 
i evealed 


theories were T 


i reporte re ds 
observation reported her 
Jmproxce 


a: Interesting bur not entirely new repone: 

WAY he avoidance performanee of whe dogs contine? Pe an 

Alter reinfercement, conceived of as Shock, ceased ss NS sessio. 
is somewhat paradoxicay, WX 


of view this finding 


reasonable 
ion and was very durable. I herce- 


in their 


From one pomt 

it is entirely Peat conditioning apparently 
in this situat 
dogs showed improvement 
after the fear itself ceased to be re- 
drive of fear well estab- 


from another 
occurred. very quickly 
fore, it is not surprising if the 
behavioral response to the fear 
words, with the secondary 
fear might very W ell become more 
the influence of the secondary re- 
ape from fear, without 
1) reinforcement. The 


been previ- 


inforced. In other 
lished, the correct res 
and more precise, under 
pe) provided by esc 


ponse to the 


prompt 
inforcement ( decremental tv 
the fear itself receiving any further (incrementa 
sort of phenomenon and interpretation thereof have 


Lichtenstein ( 19502). 
a which the shocl 


This is shown in 


sanc 


ously suggested by 
is the suddenness with 


aced the shock-escape solution, 


s-avoidance 


Also striking 
solution often repl 
rtant thing about the be- 


Vig. 12-20. 
of training described by 


But, for present purposes, the most impor 


havior modification produced by the type 
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Solomon & Wynne is its persistence, often for several hundred ies 
without a single readministration of shock (sce Solomon, Kamin, & 
Wynne, 1953). There are several factors that may enter into the cil 
planation of this finding. One is that the shock used was as intense 
as it could be without locking the dog's muscles so they. could not 
move. One would naturally expect such a shock to be Sienna 
with lively apprehension, for a long time. Then, too, it will be notec 
that spontaneous, interval responses were excluded in the Solomon- 


100 1 

50 42 
E x 
8 30 33 . 12-20. "An individual acqui- 
ə $ sition curve for a fairly typical 
S E subject. Note that the transition 
> 3 : avoid- 
= p from escape responses to : 
E 5 ance re sis final and com- 
$ 10 à 10)2— ance responses is i 
= 2 plete, with no reversals. Here 
E z i A Uu arn- 
a 5 20 8 an instance of avoidance learn 
E E NE use of irreg- 
8 ing which is not a case o g- 
3 ular or a periodie reinforcement 
E 3 *enüe, LOSS E D 
2 (Solomon and Wynne, 1953, p. Ó 
io | 1 

3 50 
! 100 


0 10 20 30 40 50 60 
Trials 

Wynne procedure 

trials. Dr. Solomon, 

he feels this feature 


e . g tween 
by the "gate" which was kept closed betwee 
e » : . indicate yat 
m a personal communication, has indicated tl 


MASH i ; sechs 
of the procedure was quite important in produ 
ing the high stability of the 


in this way: If, in 
make the "correct" 
sponse mav be follow 
trial involves the 


avoidance behavior. Perhaps it operated 
an avoidance-learning situation, the subject can 
response between trials, this means that the re- 
"ed, perhaps quite soon, by a new trial. The new 
occurrence of at least the 
incremental reinforcement), 
of the shock ( primary 


danger signal (secondary 
and it may also involve the occurrence 
-incremental reini 
would expect the effect upon the 
pressing one; so it may be 


RU : To case. WE 
inforcement). In cither case, 
response in question to be a de- 


that what the Solomon-W ynne procedure 
did. actually, was to prevent the "punishment" of the correct re- 


sponse w hich almost inevitably occurs to some extent (in a sai? 
interval schedule of trials) when interval responses are not preventec 
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Therefore, since the leaping response, with gate open 
ariably rew arded and never punished, we might 
al stability and persistence (cf. Murphy 


from occurring. 
and light on, was inv: 
expect it to have, perhaps, unusu 
& Miller, 1958). 

Just how lor 
tinue to "obey" € 
bv the change in illumination and the 
known. But it is clear that this type of learning is very resistant to 
so much so that Solomon and his co-workers incline to 
“partial irreversibility” about such learning. 
1947) have reported long-lasting effects 


ag dogs trained in the manner indicated would con- 
he "command" to leap over the barrier, given to them 
opening of the gate, is not 


extinction, 
the belief that there is a 
Moreover, Hunt ef al, (1941. 
ably traumatic starvation in rats, which would also seem 


“partial irreversibility.” There is, however, 
still some doubt about these findings (cf. Littman, 1956; Levine, 1956; 
Morgan et al, 19435 Scott, 1955). And in the case of Solomon's find- 
ings the possibility is not excluded that fear in the experimental situa- 
tion is subtly reinforced by contact with the experimenter outside 
that situation. In. the barricr-lcaping experiment, the experimenter 
arby; and it is perl 
dogs interpret the appearance of the 
mand from the experimenter. Outside the experimental situation, the 
nter presumably has considerable. contact with the subjects, 
bject relationsbip; and that relationship is probably rc- 
time the dog questions it! If, therefore, the ager 
rimental situation is indeed perceived as a “com- 
nter, and if the experimenter gives and, on 
ation, it 


from presum 
to support the notion of 


haps not unreasonable to assume that the 


stands ne 
danger signal as a sort of com- 


experime 
in a nuster-st 
inforced every 
signal in the expe 
from the experime 
rcinforces commands outside the experimental sit 
forcement would carry over (generalize) 


mand" 


on, 
unlikely that such rein 
rimental situation and produce there what seems to bc 


“obedience.” Might it not be, therefore, that the 


occas 
is not 
into the expe 


virtually. permanent 
ance to extinction reported is in reality just an 


ailure—failure to discriminate, that is, be- 


phenomenal resis 
discrimination f 


instance of 
ands given in the 


tween comm 
a given point) 
and reinforced in 


experimental situation which (beyond 


are not going to be reinforced and commands given 


all other situations? * 

?UDhe reader may here recall the remarkable resistance to extinction. recently 
Zimmerman. (19572) for rats in a double-intermittent-reinforcement 
than shock was the primary reinforcer. Here more 
a bar-pressing response, established. and maintained 


reported by 
situation, where food rather 
than 2,000. performances of 
in rats exelusively upon the b 
the effect (as W 
a deliberately fostered 


asis of secondary. reinforcement, are not uncommon. 


zut here, again, e have seen in Sections V and VIb is apparently 
“failure of discrimination”: the failure, 


dependent upon 
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As far as the Solomon findings are concerned, these inferences are 


entirely speculative. But this line of thought is empirically supported 
by a striking and now well-authenticated experimental phenomenon. 
Several years ago J. S. Brown told the present writer of having ob- 
tained the following effect. A rat was placed in one end of a run- 
way about six feet long and, through the grill floor, was presently 
shocked until it ran to the opposite end of the runw av and escaped 
into a small “safety” box. After one or two repetitions of this pro- 
cedure, the rat would, understandably enough, scamper down the 
runway and into the safety box, without the shock having to be ap- 
plied. Such behavior may occur for a good many trials, but eventually 
it extinguishes. If, however, a section of the grill at the end of the 
runway, just in front of the safety box, is kept permanently electri- 
fied, the running may never extinguish (cf. the Gwinn experiment, 
discussed in Chapter 11; also Hurwitz, 19562). That is to say, it may 
never again be necessary to shock the rat in the starting end of the 
runway as a means of making it run to the safety compartment. ‘This 
effect was later confirmed in unpublished work by Barbara Lyndon; 
and Whiteis (1955, 1956) has 


recently studied it more systematically. 
Whiteis was able to show, fairly conclusively, that the effect is due 
toa failure of discrimination. If the whole interior of the runway 
is homogeneous in appearance, i.c, if the beginning end of the run- 
wav looks very much like the Opposite end ( where shock is experi- 
enced), then the fear conditioned to the shock end of the runway 
readily generalizes to the beginning end; and cach time the rat is put 
into the beginning end, it is so afraid that it is driven out of that 
end, in the direction of the safety: Compartment, despite the fact that 
its route takes it over the energized portion of the floor grill. When a 
partition, with a small doorway in it, a 
separate the starting segment and the shock segment and when the 
EG compartments thus created are “decorated” differently, the other- 
wise observed “functional autonomy" t 
running—provided, of course, that shocks are no longer applied in 
the starting compartment. In other words, when the two compart- 
ments arc quite visibly different, the subjects 


is put in the runway so as to 


‘breaks down and the rat stops 


are much morc prone to 


that is. to discriminate between acquisition (occasional primary reward) and ex- 
tinction (no primary reward). Since this technique. of making the boundary 
between acquisition and extinction Vague was not employed in the Solomon- 
Wynne study, it is conjectured that the persistence of punishment for disobedience 
(“discipline”) outside the experimental situation may h 


ave “spilled over" and 
produced unusual resistance to extinction therein 


GENERALIZATION, DISCRIMINATION, AND SKILL 487 


“reality test," ie, to wait and see if the shoc 
partment is “still there,” than when the two areas are visibly just the 
as a result of such testing, rats find that they are now 


< in the starting com- 


same; and if, 
actually safe in the starting compartment, then, sensibly enough, they 
when they cannot distinguish between the 


stop running. It is only 
f the runway that they get into and 


starting and the shock areas o 
persist in the “vicious circle.” 
We conjecture, therefore, that the “partial irreversibility” of the 
fear conditioning reported by Solomon & Wynne was of an analogous 
ceter. In the Whiteis experiment, fear in the starting area of the 


cl 
“irreversible” because the animal's fear 


alley 


proved to be apparently 
was getting reinforced in the adjacent area and was generalizing back 
into the now actually safe arca. It seems entirely possible that the 
discipline which the Solomon-Wynne dogs presumably received every- 
where celse generalized, in like manner, over into the experimental 
p ally, the dogs might learn that the experimenter no 
longer “meant it” when the command (light change and gate open- 
ing) to jump came in the experimental situation; but since this com- 

5 reinforced so powerfully and since all other com- 


mand was initially 
mands outside the experimental situation were consistently enforced, 


it would presumably take a very smart (and brave) dog indeed to make 
ination. Is it not suggestive that no one has reported simi- 
"obedience" (resistance to extinction) in the shuttling 
anisms which are ordinarily subjected to 


situation, Eventu 


such a discrim 
larly persistence 
of rats or other org 


behavior à ) i j 
ance) training only in the experimental situation? (See 


obedience (avoid 
also Chapter 6, Section VI.) : , 

Dollard & Miller (1950). in attempting to give a learning-theory 
human neurosis, have made much of the hypothesis (also 
accepted, though differently expressed, by Freud) that pathological 
adult fears are simply fears that have been learned by the individual 
in infancy and persist because the individual fails to realize that “con- 
ave changed” and that, under the circumstances of adult life, 
free to do certain th 
abooed and punished. The evidence reviewed in the 
also Mowrer, 1960, Chapter 10) suggests that this is 
Conceivably there are some instances where an 
unnecessary adult fear is kept alive by generalized reinforcement 
(Solomon ‘example) or by a vicious circle (Whiteis example). How- 
psychopathology the individual's guilt seems real 
His fears would therefore reflect a failure far 


account of 


ditions h 
he is now 
were severely t 
preceding pages (see 
an oversimplification. 


ings W hich, in infancy or childhood, 


ever, in human 
rather. than spurious. 


more basic than one of mere discrimination. 
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XII. Discrimination, Skill, and 
; 1 H s 
“Response Generalization 


Throughout this book we have stressed the thesis that : Rectal 
does not involve motor learning, i.e., the strengthening of Be i e 
connection between some stimulus or drive and a particular ath 
muscles, Instead, "habit" is conceived as a matter of the hopes anc 


iti i i at are associated 
fears that have become conditioned to the stimuli that are assoc 


with the occurrence of a given response. And the evidence for suen 
a thesis is already substantial and apparently growing. acin 
with some justification it can still be asked if there are na : wes 
some instances of purely “motor learning,” particularly in the doi H 
of the so-called skills. It will be pointed out that the violin iia i 
the Olympic hurdler ust have his muscles “trained in special w a 
In a sense this is, of course, true, Many skills call for unusual, not ; 
say prodigious, muscular d velopment; but fitness and strength a 
muscle is not what we mean by learning—though "fitness" is EN i 
tedly an essential aspect of many skills. The learning aspect of e 
problem has to do with the co-ordination and delicacy of the contro 
which the individual has over 

And how, 


his musculature. 37. 
we must ask, is such control acquired? It is, we conjec- 
ture, a matter of increasingly WEA 
and athletes have to study with 
coaches; and much of this tr: 
what is right and wh 


refined discrimination. Most musi 
and be trained by special teachers anc 
aining consists of intensive instruction in 
at is wrong about their performances. From Fon 
kind of tutelage they themselves get a proper feel for what is correct, 
what incorrect. Some of this learning is "motor" in the sense that 
it involves discrimination betw cen the proprioceptive feedbacks no 
correct and from incorrect responses; but the process is broader 
than this. The hurdler can see, as well as feel, some of his movements; 
and, in the case of the Violinist, to kinesthesis, sight, and touch is added 
sound. No amount o “purely motor" training will ever make a 
Kreisler out of a person with “no car"; and we may conjecture that 
champion hurdlers need certain other forms of sensory acculty, as 
well as strong legs and lungs.® 

It will be apparent that this 


interpretation of skill is not merely 
consistent with but is virtually 


demanded by the revised two- 

position, Other writers (e.g., Keller, 1954; Bahrick, Fitts, & Schneide 
"For a remarkably explicit, earlier statement of this hy pothesi 
Vol. I, p. 4927. where we read: “Professor I 
curacy with which a certain tenor s; 


q see James (1890. 
i f. ac- 
Beaunis (1889) found that the a 
i COT ere 
ang was not lost when his vocal cords w 


idi re not 
made anaesthetic by cocain. He concludes that. the guiding sensations arc 
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1955; and Gibbs, 1954). 
advanced a very similar type of interpretation. 

Finally there remains to be considered in this section the concept 
of “response generalization.” It is referred to as a concept, rather than 
on that, as we shall attempt to show, 
of stimulus generalization. Such a 


on quite independent grounds, have recently 


as a phenomenon, for the re 
it probably reduces to a matter 
| the premise that psychologically a response 
is the stimulation which it produces, in the actor and in others. Thus 
one response is like or unlike another in terms of the stimulation 
is contingent upon its occurrences; and we would expect, ac- 
that so-called. response generalization would follow the 
erning stimulus generalization, 

zation, has been applied to the fact that 
rewarded for making a particular re- 
not only this response, but also re- 
ans the "same." Sheppard 


conclusion follows fron 


which 
cordingly, 
anterior principles gov 
‘The term, response general 
an organism which has been 
will tend. to reproduce, 

are similar but by no me 


sponsc 


sponses which 
(1958) puts the matter thus: 


lition to stimulus generalization, an analogous phenomenon of 


response generalization is sometimes supposed to operate so that (3) à 
stimulus to which one response has been conditioned tends to evoke other 
and (b) the magnitude of this tendency (for any particular 
sponses) is governed by the dissimilarity between that 
and the response originally conditioned. Although this principle 
ralization is also of considerable theoretical importance, 
le toward the quantitative determination of 
case of stimulus gener- 


In adc 


responses, 
one of these r 


response 
of response gene 
even less progress has been mac 
the shape of the gradient in this case than in the 
alization ( pp- 242-243)+ 

neralization is the antithesis of 
ation of some standard pat- 
id to 


a sense, therefore, response ge 
alls for a very precise replic 
tern of action (and return stimulation). It may, in fact, be 
t the usual, normal tendency of living organisms prior to 
because the secondary reinforcement. that is 


In 
skill, which € 


represen 
highly specific training: 


resident in the laryngeal muscles themselves. They are much more probably in the 
on the role of discrimination and emotional 
is not fully apparent, it may 


car (p- 253). 

TIF the cogency 
ng in the develo 
to recall how 


a familiar selection; or, to. take 
are made by hearing distorted, labored, defective speech. 


vis skill; and when we hear the grossly (or even 
r speech. of another, we feel some of the dis- 
own. For a review of 


of our emphasis 
pment of “motor skills” 
a well-trained musician is affected by hearing a tyro 
a more general example, how uncom- 


conditioni 
be helpful 
“murder” 
fortable most persons 
We are all highly trained in 17 
slightly) inadequate and imprope 3 
comfort thar would occur if the performance were our 
] practices of speech correctionists in this connection, see Mowrer 


the theories and 


(1958). 
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“habit” generalizes from the stimuli produced by the response which 
has been selectively rewarded to stimuli produced by more or less 
similar responses, performance has an initially nonprecise, general, 
even “sloppy” (but not unadaptive ) quality. (Only if such generaliza- 
tion occurs will the subject feel rewarded—have a sense of "accom- 
plishment"—on the first. several repetitions or approximations of the 
correct response.) With specific nonreinforcement (or punishment) 
for similar but not (by some criterion) identical responses, these tend 
to drop out; and performance becomes more precise, artistic, "dis- 
criminating." * 

Here we scc, once again, converging evidence of the power of the 
new two-factor position and the scope and variety of its implications. 


XIII. High-Speed Performance and 
the Problem of Control 


However, one technical difficultv, or at least objection, remains. 
A good many years ago Lashley (1917; sec also Lashley, 1951) au 
vanced what may seem to be a rather specific argument against Hie 
feedback conception of habit. He pointed out that in rapid typing or 
performance on a musical instrument, sequences of finger movement 
may be executed so quickly that there is literally no time for im- 
pulses from one movement to get back to the brain before the nest 
movement is initiated, And from 


types of learned, “habitual” 
volving m 


this observation he argued that SOIS 
action are really patterned sequences 1n7 
any predetermined, pre-assembled (Hebb, 1949) individual 
responses and that such a conception of habit demands the postula- 
Tt of purely "central" mechanism, rather than the 


mixed central-peripheral mechanism involved in the feedback arrange- 
ment. 


tion of some so 


sgot s i iscussi : scurring 
Osgood (1953), in discussing the notion that responses occurring 


is 


^ [t is not without interest that Sheppard (1958), although operating on the b 
of rather different theoretical presuppositions, believes that “considerable evi- 
dence can be adduced for the Proposition that the gradients of stimulus and re- 
sponse generalization both conform to Ithe same] “exponential decay function" 
(p. 243). This is another indication thar, i 


: ith 
at bottom, we are here dealing witl 
one and the same phenomenon. 


It will also be evident that in response generaliza- 
M PES : with what has traditionally been termed “transfer 
of training? or. simply, “transfer.” At a recent professional meeting, Bugelski. 
Goss, Murdock, Perkins, & Wickens (igs o 

ing processes in transfer.” which also, in effect, reduced response generalization 
to stimulus generalization (see also Mow rer, 1960, Chapter 2). P 


tion, so-called, we are dealing 


8) conducted a symposium on “Mediat- 
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in rapid succession become “chained” together by means of “cues 
produced by preceding movements," puts the matter thus: 


This behavioral analysis of the development of instrumental skills, while 
undoubtedly true in part, is certainly not a sufficient. explanation. It relies 
heavily upon the sensory “feed-back” from responding muscles, and 
Lashley (1917) long ago pointed out that there simply isn't enough time 
between the components of a rapidly executed skill. for signals to be 
relaved back and forth over the motor and sensory nerves connecting 
brain with periphery. He used for illustration a pianist plaving a swift 
cadenza. It seems probable that in the final stages of skill formation cells 
in the central motor areas become organized in such a wav (cf. Hebb, 
that the time and ordering of sequential movements is determined 
of the necessary conditions for the development of such 
zation is probably a fairly rapid sequence of move- 
sensory feed-back mechanism. We need a great deal 
s on these points (p. 401). 


1949) 
at this locus. One 


central motor orgar 


ments guided by the 1 
more information from the neurophysiologis 


ms were directed against a notion which 


Actually, Lashley’s criticis 
that involved in our present conception 


is somewhat different from 
of habit. Lashley was attacking thc then prevalent view that one overt 
response can become directly "conditioned" to stimulation provided 
by the immediately prec 
become 


eding overt response and that long scrics of 
“linked together" in this manner, so that, 
irst response in the series, the other re- 
without any sort of central 


responses may 1 
c occurrence of the f 
ponses will be automatically. "run off," 
motivation, direction, or control. For ri 
we do not accept the view that overt actions are directly. conditiona- 
ble and would join Lashley in rejecting the chain-reflex, or “chaining,” 
theory of serial learning. However, as Osgood indicates, in admitting 
central factors into the picture, we are not at all obliged to exclude 


of influential sensory feedback. 
who is both a neurologist and an 


given th 


asons previously considered, 


the possibility 
Fortunately. Quastler (1953). 


engineer, has made a comprehensive analysis of this problem which 
es it in a manner cot 


In presenting his analysis he says: 


resolv agruent with the assumptions of two-factor 


theory. 
ains a series of studies of human performance in informa- 
tion transmission. We have estimated the rates at which information is 
transmitted by people reading, typing. playing the piano, doing inane] 
arithmetic, or assimilated by glancing at a display of letters, playing cárds, 
scales and dials. In these situations, we have attempted to establish the 
principal limiting factors. However, our primary interest is not in such 
things as measuring the number of "bits" per second which a typist can 
transmit from a script to her keyboard; in fact, we are not primarily 


This report cont 
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interested in information transmission. per second. Rather; our A 
concerns human performance in the general activity of monuu ! T 
essing; this includes, besides transmitting, activities such as Pin 
filtering, remembering, correlating, learning, selecting a response, ete. (p. 5)- 


Symbols 


Input 


Sensation 


Recognition 


Planning of 
action sequence 


Motor action 


Output 
Sounds 


Error detection 


Fig. 12-21. Schematic diagram 


ae ; sequential 
of component processes of a "simple seque! 
t 


8 —music. reading —from. Quastler (1955, p. 66). 


It will not be possible here to go into the details of the Quastle 
analysis, but the 


E Feds fi he 
general tenor of his thinking can be had from t 


i P oe ^ H ; [ his 
following excerpt and a diagram taken from the last section of | 

à HE do M CLEA t : A è DroCCSS- 
report, entitled “Implications of Mechanisms of Information Proce 
ing." 

. {In Fig. 12-21] the sight-reading process, . . . is pictured diagranimat- 
ically. The physical events 


i i ands 
1 (scripts and sounds) are projected. on banc 
| which run diagonally, left to right]; the boxes represent mental processes 


or rather, the abstractions here used to describe areas of mental activities: 
each box repr sents a complex system of action. The diagram contains 
the minimum of components needed to represent the flow of information. 
In particular, it applies only. to over-learned activities; no allowance 1S 
made for feedback or other checks during any action. It is vastly sehe- 
matized, assuming the system to be synchronized and leaving out con- 
nections between chains which undoubtedly exist. Thus, the diagram Js 
supposed to represent a simplified. version’ E 
sight-reading. We are not I 
varied activities; we onh conjecture that, 
learned virtually to. perfection, 
ought to be efficient. The whol 


of the activities involved i 
"oposmng to study the organization of these 


i ¿is 
secing that the total task e 
the organization. of component pg 
€ process involves several diverse acts 9 
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information-processing. What we evaluate is only the over-all result; we 
have not taken account of the information processes in planning coordinated 
action nor of that used in the feed-back loop. Accordingly, the starement 
that a pianist has been shown to transmit about 22 bits sec does not imply 
ss information at only this rate; it only means that the 
net result of his activities could be’ paralleled by an appropriate ideal 
transducer with a capacity of 22 bit sec. This observation sheds some light 
on the main result of this study, the finding that the information measure 
is relatively invariant over a cons iderable range of conditions. This phenom- 
enon could be due to some aspect of the clementary processes involved, 
but it could equally be an expression of the way in which elementary 
ation-handling processes are efficiently organized into a complex 


that he proces 


inform 
system (pp. 66-67 )- 

Clearly, the problem of rapid sequential activity is an intricate onc. 
original objections to the chain-reflex interpretation seem 
but the same objections do not apply to the revised 
formation and execution. Many de- 


Lashlev’s 
well founded, 
two-factor conception of habit 
to be worked out as far as this type of interpretation is 
as the Quastler analysis indicates; but as far as can bc 
scen at present, there is no known neurological fact or cybernetics 
principle (see Mowrer, 1960, Chapter 7) W hich às in opposition to the 
assumptions demanded by our theoretical position. 

No extended analysis of these problems will be undertaken; but 
it should be pointed out that as speed of performance increases, ac- 
of control tends to decrease. And where responses are being 
rates, "control" in one sense of the term 


tails remain 


concerned, 


curacy 
emitted at extremely high 
Iy lost in that “mistak 
‘annot be “prevented.” 
to "check" his behavior (both in the sense of in- 


" cannot be corrected before thev 


is complete 
occur, i,c., Cant 


has an opportunity 
if need be, stopping 1t) W hile it is in. progress. This 


specting it and, if ne 
advantage 18 embodied in many aphorisms, such as “Make haste 
dS and “Easy does it.” One of the hallmarks of virtuosity and 
“brilliance” is the very capacity to do things rapidly avd perfectly. It 
is not that there is no feedback from such behavior but that it arrives 
te to be useful as far as the control and correction of any given 
avior is concerned. Hence Lashley and Hebb are probably 
ning that extremely rapidly sequential performances must 
“organization” not required when the pace is 


‘In more leisurely performances one 


too la 
item of beh 
right in assul 


presuppose a form ot : 
morc leisurely; ^ vet this fact in no way mpugns the notion that 


(A recently (and still incompletely) reported study by Murdock (sce Bugelski 
et al, 1958) suggests thar the speed of sequential actions may depend, after all, 
upon discrete reaction times. If this should prove to be the case, much of the 
force wf the Lashley-Hebb type of argument w ould, of course, be lost. 
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“habits” are, primitively and basically, a matter of the hopes and fears 
that are conditioned to the sensory (including the imaginal—see 
Mowrer, 1960, Chapter 5) consequences of their performance. 

Sometimes the term “habit” is thought of as applying mainly to 
well-practiced, semi-automatic, and only partially "conscious" actions, 
in distinction to behavior that is said to be fully conscious and “volun- 
tary.” It must be admitted that some sequences of behavior run off very 
smoothly, whereas others involve hesitation, choice, and correction. 
But the difference is apparently one of degree rather than of kind." 
As indicated in the preceding chapter and elsewhere in this book, the 
important and basic distinction is rather between all such behavior of 
this kind (involving the skeletal musculature) and the emotions, which 
are truly conditionable and occur outside the sphere of conscious, 
voluntary control, But, somewhat paradoxically, it is these involuntary 
reactions and the conscious states they produce which then mediate 
and motivate the process of choice and volition. 

This analysis is carried further in the sequel to this book, Learning 
Theory and the Symbolic Processes (Mowrer, 1960). 


» Since the above was written, a colleague (who is a bic ling enthusiast) has 
provided the following report of a remarkable experience: "Recently, as a result 
of a slight mishap, I found that my bievele w i 
now to the right, now to the left, and so on. 
that the front anc 


ard to guide, tending to ‘draw’ 
amination by an expert rev caled 
3 back wheels were out of alignment (were not ‘tracking’)s 
and it was suggested that if I would just keep riding the bike I would probably 
‘get used to it and no longer notice the steering difficulty. However, the difficulty 
continued to be distracting, and I ordered a new fork for the front wheel. But, 
sure enough, by the time the fork arrived, | was aware of the problem only 
occasionally; most of the time the bicycle seemed to be quite all right. Then 2 
surprising thing happened: the new fork i ii 
after I again startec 
my bands—were a 


was installed and for some minutes 
1 riding the bicycle 1 noticed that my hands—I say, not ‘I but 
ternately turning the handle-bar slightly to the right, slightly 
to the left, etc. Immediately I realized what had hap ened: | had learned to 
compensate for the misalignment and therefore bec 
although the misa ignment had now been corrected, | was still, automatically 
and unconsciously, compensating. Of course, this idu usos ‘behavior soon 
disappeared; but, for a time, it provided an unusually clear and striking example 
of the development of an ‘unconscious’ skill or habit. As already indicated: I had 
originally been only too vividly aware of the "trouble; bur I had no knowledge 
at all of the solution I had adopted, i.e, what I had itched to do about it.” Ic is 
unclear at present whether this type of observation can or cannot be reconciled 
with the general point of view expressed in this chapter—and in this book as a 
whole. Related problems are discussed elsewhere i i 


ame unmindful of it; and, 


(Mowrer, 1960). 
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Developmental psychology, 303 
Disappointment, 204, 211 
and appetite, 204 
cue function of, 469 
followed by reward, 469 
and hope, 163 ff. 
and inhibition, 295 
as loss of hope, 407 
asa punishment, 164 
rclation to anger and sorrow, 168 
r, 407 


, questioned by Bitter- 


and return of fes 
Discipline, 486, 4 
Discontinuous rei 
Discriminant operant, 207 
Discrimination, 37-39: 438 tT. 

CR method of study, 450 


nforcement, 458 


as cognition, 450 

and conditioning, 36-37. 204-295 
and cues, 450 

and delayed reinforcement, 
easy and hard, 360, 448 
failure of, 485-486 

and fear generalization, 421 
insoluble, 412-413 

of motion and rest, 479 
and motivation, 198 
rrowing of generalization, 441 


357-360 


ana 
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Discrimination (continued) 
and paradoxical learning, 434 
and secondary reinforcement, 133 
and skill, 488 ff. 
Skinner's method of studying, 442 
to study sensory acuity, 446 
theory of, 452 
rimination hypothesis, 457-458, 462, 
469 
alternative to, 459 
cognitive version of, 474 
confirmation of, 463 


and counterconditioning, 473 ff. 
dramatically confirmed, 475 
more support for, 467 
noncognitive version, 468 ff. 
objections to, 469 
Discrimination problems, series of, 452 
Disgust, 417 
Dismay, 164 
Disorganization, 197, 237 
Dissimilarity, of CR and UnCR, 73 
Distraction, 338 
Docility, 405 
Double alternation, 328 
Drive, 186 ff. 
acquired, 49 
and activity, 272-273, 304 
without activity, 304 
and arousal, 163 


cyclic nature of, 145 
definitions of, 163, 200 
and emotional arousal, 163 
exploratory, 175 

externali 


tion of, 109, 154-156 
generalized state of, 157 
increment and decrement, 273 
induction, 87 
intensity of, 273 
measurements of, 163 
not a stimulus, 272 
reduction of, 68, 87-88 
and reinforcement, 200 

Drives Motives), 

components of, 184 


(see also 


affective 


emotional, 


«307 


as energizers, 271 


equivalence of, 153 
and external stimuli, 181 


Harlow’s third category, 173 
inferred from behavior, 173 
interaction of, 158 
irrelevant, 158, 162 
metabolic, 307 


metabolic 
not all alik 
onset of, 145 


ad noxious, 149 


versus needs, 187 
primary and secondary, 
as reinforcers, 273 


tautology in defining, 173 
visceral, 
Drive fear, 163 


and drive equivalence, 156 tf. 
generality of, 157 
Drive increment, 418 
and conditioned motivation, 146 
Drive induction, 319 
and pain, 275 
Drive reduction, 186, 303, 305, 319 
abrupt, 85 
and conditioning, gy 
contradictory implications, 202 
gradual, 83 
and incentive induction, 201 
and learning, 193 
and pleasure, 203, 275 
produced by brain stimulation, 207 
and release of action, 
and 
455 
Drive-reduction hypothesis, 207 
Dynomogenesis, 162 


secondary reinforcement, 104 


Fating, in absence of hunger, 189 
as habit, 195 
increased by electric shock, 159, 16! 
increased by fear, 161 
to reduce fear, 162 
Education, 4 
Effects, of behavior 


T 
and control of behavior, 327 
Effort, 399 
and extinction, 234, 401 
as an inhibitor, 436 
Emotion, and perception, 279 
as secondary drive, 78 
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Emotional conditioning, a delicate pro- 
cedure, 396 

Emotions, 40. 48, 59 
basically reflexive, 310 
and behavior, 62 
as behavior selector: 
conditionable, 392. 416 
counterconditioning of, 417 
different from behavior, 416 


distrust of, 308 

ecology of, 168, 271, 2957 
oT. 47 

extinction of, 403 

and fatigue, 402 

as forecasters, 61, 169 

sification, 167-169 


296, 392. 404 


four-fold c 
and gains or losses, 169 

as goal events, 4098 

identification of, 167 

and intentions, 407 

as intervening variables, 60, 404+ 416 
involuntary, 416. 494 

we theory of, 48, 407 

ack, 418 

f 


James-Lange 
no kinesthetic feedb 
and law of effect, 416 
and motions, 418 

as motives, 273+ 275+ 494 
not instrumental, 417 
not intelligent, 308 

as positive and neg 
prospective, 165-166 
psychology of, 61-62 
and reinforcement, 417 
situational definition of. 


itive valences, 31! 


166 ff. 


theory of, 169 


as wasted reactions, 308 


“mpiricism, 19! 


înjoyment, 186-187 
interest in, 349 


nvironment, 
knowledge of, 34° 


Evolution, 2. 17 
and behavior, 17> 385 
theory of, 260 


“xcitation, 40 
optimal lev cl of, 1 

"xcitement, 190 

200 


85 


appetitive. 
and heightence 
"Npectancy. 59. 32? 


Į activity 196 


pectation, 287, 324-325 

and disappointment, 477 
perimental neuroses, 432 
Experimenter, responsibility of, 296 


Exploration, 172, 178, 22 
and fear, 176 
visual, 17 

inction, 388 ff. 

apparent failure of, 481 ff. 

conflict theory of, 399 

as counterconditioning, 390 ff, 403, 
419. 469 

under curare, 403 

and effort, 233. 400. 428 

fatigue theory of, 398, 402, 473 


77 


fear of, 55 

formula for, 403 

and frustration, 43, 424 
and generalization, 440. 445 
and habit, 56 

with irrele 
ad motivation, 231 


nt stimuli, 235 


nature of, 398 

and negative practice, 410 
new theory of, 409 
nullified, 479 
without practice, 
srevention of, 111 
and reality testing, 403 

despite reinforcement, 427 
reinforcement theory of, 428 

108, 227. 23» 374 375 


37 


resistance tO, 
457 

a reversal of meanings. 427 
of secondary reinforcement, 118 
and sensory feedback 
theories of, 426 

and time, 394 
unusual resistance to. 484. 485 


with and without S", 233 
Extinctions, successive, 451 


Facilitation, and inhibition, 319 


Faculties, 20 
Faradization, $ 
sensation of, 243 
of the sigmoid gyri, 239 
as UnC 


542 SUBJECT INDEX 


Fatigue, 72 pathological, 487 
and anger, 410 and physiological need, 149. 204 
as cause of frustration, 432 psychology of, 398 
and conflict, 400 punishment of, 418, 421 
as countermotivation, 402 and reality testing, 176 
and extinction, 399 reduction of, 32. 53 
and frustration, 409 reflexive, 135 
and inhibition, 401 reinforcement of, 85, 122, 416 
limited to muscular responses, 402 relaxation of, 138 
not essential to extinction, 403 and relief, 163 
Fatigue theory, of extinction, 403, 473 a response-produced drive, 77 
Fear, 28 ff., 48 ff. return of, 164 
acceptance of, 60 rise in, 203 
and anger, 407 and secondary reinforcement, 34. 95 
an aversive emotion, so, 122 self-produced, 177 
as cause of curiosity, 175-176 superfluous as a concept. 435 
as à concept, 57. 122, 138 after sympathectomy, 240 
as conditioned pain, 93, 127 systematic importance of, 30, 76. 95. 
conditioning of, 24-26, 38, 57 (see also 403 
Fear conditioning) of thirst, 157 
of continued pain, 133-135 and thumb-sucking, 394 
counterconditioning of, 395 unconfirmed, 482 
and curiosity, 29-30 vague and unlocalized, 176 


in the dark, 181 and visual deprivation, 180 


as a drive, 34, 93, 95, 97, 435 ar conditioning, 26 IT. 393 
and drive onset, 81, 88, 124, 188, 319 generalization of, 446 
and other emotions, 211, 392 


with intermittent reinforcement, 473 
escape from, 37 


with metabolic drives, 145 ff. 
and exploratory drive, 176 to a situation, 132 


extinction of, 372, 397 to weak UnCS 
of fear, 133 E 


130, 132 
ar reduction, 479 
conditioning of, 128 
and frustration, 96 


and frustration, 407 

generality of, 162 

generalization of, 38-39, 143, 393. 421, as reinforcement, 97 
487 P ; 
and hope, 59. 150, 394 

of hunger, 15 


:edback, concept of, 309 
and delay of reward, 383 
delayed auditory, 245 ff. 
and extinction, 426 

and habit, 246, 280, 286 
informational, 220 


161 


not inhibited by punishment, 419. 422 
and inhibition, 295 
an innate reaction to pain, 129 


intensi 22 Mie" 
intensity of, 172 labyrinthine, 299 


as an intervening variable 3i. ds negative, 201 


13404 


Fe. UR a 
learning of. 34. 86 
mastery of, 177 


positive, 201, 202 

proprioceptive, 234 
and retroflexive bac 
Sensory (se 


i kick. 320 
and metabolic drives, 145 ff. ick. 3 


nonestinction of, 482. 487 


also Sensory feedback). 
19$. $37 
visual, 298 
Feeling, 2 


and the organism, 79 
overcome by hunger, 433 


) 
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Field expectancies, 322 

Field forces, 311 

Field theory, 306 ff. 
acceptance of, 342 
adumbrations of, 314 
Hull's analysis of, 342 fF. 
and pli ice learning, 331 


and retlexol 
and S R psyc iology, 313 


^. 306 


Fitness, 488 
Fixation, of be ior, 336. 405 
following frustr ation, 412 
and intermittent reinforce 
and rigidity, 414 
on simpler La is 4 
Foresight, 18, 149.3 
Fractional goal 
23,363 


Free operant, 5 


ment, 415 


Freezing, 56, 88 
Frenzy, 034 
V requeney, law of, 35° 
Frustration, 388 ff. 
and anger, 165, 403 
and behavior variation, 412 
biphasic theor of, 425 
cue function of, 469 
as a drive, 96 
and fatigue, 409 
and fear, 407 
and field theory 
and habit elimination, 41 
and habit strength, 472 
and inaction, 4! 
and loss of interest, 405 
notional interpret 
opposite of hopes 496 


and other emotions. 168 
intermittent 


312 


nonet ations of, 405 


reinforce- 


reduced by 
ment, 4697479 


and regression, 413 
to extinction, 408 


nent, 227 


and resistance 
ry reinforcer 


and seconda 
ach abandoned, 405 


SR appre 

theory of, 404 

and threat, 425 

as threat, 168 
Vrustration-fixation hy 

alternative inte rpretat 


pothesis, qui ff. 
ion, 415 


re "E FE 
Frustration theory, Skinner-l lumphre 
effect, 477 ý 
Frustration tolerance, 413. 470 
Functional autonomy, 482, 486 
Functionalism, 3, 4. 14 
Future, perception of, 367 


Galvanic skin reaction, 402, 421 
Generalization, 37 ff., 438 ff. 

and aperiodic reinforcement, 443 

backward, 415 

biological adv antages of, 439 

correction of, 38 

curve of, 394. R9 H4 

and discrimination, 37 fiar ff. 

Wegen of. 39. 442 

of fear, 28, 38-39 

fixed or modifiable, 441 

from image to object, 282 

and learning, 44° 

mediated, 442 

once-always, 439 
one-many, 439 
through similarity, 362 


temporal. 362 
unrealistic, 38. 442 
Gestalt psy chology, 1 

Gestalten, 324 
Gestalttheorie, 13. 18 
Goal, alternate routes to, 336 
Goal events, three types of, 408 
Goal g lient, 342+ He 348 
Goal reaction, 105. 
Goals, change in, 413 
positive and negativ C, 321 
Gradient, of irradiation, 342 
of gene alization, 443 
also Gradient of 
cinforcement, 345 ff. 
of reinforcement, 387 


34. 308 


(see reinforcement) 
Gradient of r 
and amount 
contraction of. 3 
duration of, 356. » 
four types of, 38. 
Hull's analysis M 364 
inadvertently bridged, 
length of. 384 
ative findings, 351 tT. 
tive findings, 354 


neg 


pos 
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Gradient of reinforcement (cont.) 
its reality questioned, 350, 358, 360 
and retraction, 358 
summary, 375, 377+ 384 
and two-factor theory, 384 

Grief, 211 


Growth function, for habit and second- 


ary reinforcement, 251 
Guiding sensations, 488 
Guilt, neglect of, 437 


Habit, based on sign learning, 225 
competing, 358 
control of, 221, 261 
bond conception of, 218, 220 

237. 361 ff. 

concept of, 16, 330, 359, 391 
derived from conditioning, 256. 280 
discriminative, 2 


and drive reduction, 124 

as emotional feedback, 488 
and emotions, 263 
extinction of, 398-399 

and field behavior, 296 


as hope or expectation, 22. 

in Hull's system, 66 ff., 112 

as an incentive mechanism, 261, 301 

and inhibition comparable, 216, 256 

and kinesthesis, 75. 296 

not localizable, 298 

logical difficulties, 
Habit theory) 

new conception of, 


361 ff. (see also 

108, 118-120, 1 
144 164, 214, 223, 246 

preformed, 227 

reinterpreted, 214 

as resistance to extinction, 22 

as response probability, 218 


as secondary reinforcement, 112, 2 


25, 


230, 233 ff., 245 
and sensory 
228, 298 

spontaneous recovery, 4oo 
not an S—R bond, 216, 267, 391 
as temptation, 271 


feedback, 216-217, 222 


Thorndike’s conception of, 16-18 (see 
also. bond conception of) 

in two-factor theory, 89 (see also, 
new conception of ) 


unconscious, 494 

and voluntary action, 298, 494 
Habit-family hypothesis, 342 
Habit formation, 18, 215 

dependent on conditioning, 390 


Habit strength, and extinction, 457 


frustration tolerance, 470, 472 
and intermittent reinforcement, 457 
measures of, 397 
negative, 398 
oscillation in, 338 
and rate of acquisition, 457 
resistance to extinction of, 225, 470 
and secondary reinforcement 

246, 465 

Habit theory, 212 ff. 
common ground with field theory, 344 
weaknesses of, 17-18, 390 

Habituation, 372 

Heart 


,23g-228, 


e, as index of fear, 421 
Hedonism, 203, 275 ff. 

and brain stimulation, 205 ff. 

difficulties with, 270 

and homeostasis, 203, 205 ff., 276 

necessarily teleological, 191, 

Young's view of, 188, 275 ff. 
I loarding, 151 (f. 

by going to food, 153 

of food, 151 

and hunger fear, 151 

motivation for, 152 

reinforcement of, 152 

theories of, 151 

of water, 151, 152 
75:1ff. 
basically sound, 210, 3 
and behavior theory 


Homeostasis, 


o5 

+305 

and brain stimulation, 205 tT. 
criticisms of, 170-171, 175. 1847185 
defense of, 201, 


disturbers of, 269 
exceptions to, 416 

and hedonism, 203, 205 ff, 
and learning, 301 


76-277 
and Olds-Milner etfect, 209-210 
and pleasure, 203 

predictive capacity of, 301 

and sexuality 
supported, 186 


Hope, and action, 199, 
pt of, 61, 312 
and disappointment, 163 

and drive reduction, 188, 199. 


conc 


and other emotions, 150, 168, 211 
59, 85, 326 


and expectancy 
and groal orientation, 197 
and habit, 295 

and jov. 433 

as used by Levin, 312 

loss of, 164 

opposite of frustration, 406 
as reassurance, 134 

as relaxation or excitement, 196 
as relict, 85, 139 

and speed of response, 475 


secondary reinforcement, 


as type-z 
101, 256.472 
unconfirmed, 482 
Hope and fear, mutual cancellation of, 
4n 
I lopefulness, 205, 263 
Hopelessness, 137+ 164. 197 
Hostility, 404 
Hunger, and 
fear of, 146, 1517153 
promise of reduction i 


and response chresholc 


activity, 198 

$2, 156.1 
n, 140 
ls, 198 


and saccharine, 193 
Hunger fear, and habit, 14 
Hypotheses, 381 


8 


Ideas, and memory, 367 
Identification, 260 


Identity, sense of, 237 


182 ff. 


sensation, ?5 


Hlumination, a 
as reinforcement, 
Image, a conditioned 
amg 

of gratifi 


ation, 260 


Imagery, 
Imitation, 174 
Immediate memory. 3 


Incentive, 179. 200 


Incentive motivation. 


261 ff. 


criticisms of, 262 
as emotion, 263 
how acquired, 266 


and re, 105 
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redundant in Hull's system, 262 
ondary reinforcement, 263 


and s 
tency, of reward and punish- 


Incons 
ment, 472 
Indeterminancy, 10 
Induction, 438 
Information, as feedback, 309, 39! 
flow of, 492 
from motor activity, 260 
and response control, 299 
transmission of, 491 ff. 
Inhibition, achieved indirectly, 217 
conflict theory of, 398 
and delay of punishment, 370 ff. 
and electric shock, 206 
through extinction, 401 
fear-mediated, 24-25, 261 
and feedback, 217 
naturc of, 398 
nonspecific, 56-57 
as passive avoidance learning, 335 
Pavlov's theory of, 40-41 
produced, 72, 401 
reinforcement, or alterna- 


response- 
Inhibition of 
tive learning, 429 
and bar pressing, 432 
and common sense, 429 
procedure, 429 


encouraged by 
new evidence for, 431 
not really paradoxical, 430 


Insecurity, from deprivation, 137 


Insight, 13 
as deduction, 439 
and Ges alt psychology. jac f 


higher types of learning, 329 
Instinct, concept of, 315 
Instincts, and learning, 3! 5 

lists of, 
Instrumental 

Conditioning. 


8 


3 


(sce also 


behavior, 426 
instrumental) 


Integration. 378 
and speed of response, 387 
temporal, 366 


Integrative learning. 379 
Intelligence. 18 
and the emotions, 308 
and retroflex action, 317 
Intentions, 407 
Interest, 3!4 
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Interests, as hopes and fears, 340 
Intermittent reinforcement, 457 tf. (see 
also Reinforcement, intermittent) 
discrimination hypothesis, 118 
double, 123 
in fear conditioning, 473 
preference test of, 461 
and punishment, 47 


really continuous, 459 
secondary reinforcement, 459 
Interval responses, 446, 448-449 
functional, 477 tf. 
physically prevented, 484 
subtly punished, 484 
Intervening movements, 286 
Intervening variables, absence of, 56, 
288 
and Behaviorism, 163 
e 


mplified by emotions, 404 
ignored by Pavlov, 258 
Volman's use of, 309, 322 

Intoxication, 270 

Introspection, 5-6, 14, 78 

Introspectionism, 6 

Invention, 174 

Invisibility, ¢ 


dvantage of, 181 
Isolation, effects of, 180 
Visual, 180 


Isomorphism, 61 


Joy. 165 
Joy center, 208 
Jumposity, fallacy of, 173 


Keller-Schoenfeld-Dinsmoor 
sis. 454 ff. 

itically appraised, 456 
Kinesthesis, and habit, 285 
Knowing, 20 


Hypothe- 


€ 


knowledge, and action, 324 
without action, 322 
as cognitions, 322, 329 


of reward and nonreward, 454 


anguage, 215. 260, 180, 418 
arent extinction, 237 

atent learning, 109. 228, 237, 267 
aw of effeer, 21 ff. 

and conditioning, 62 ff.. 89, 251 


and conflict, 390, 422 
and delayed reinforcement, 349. 353 
and emotions, 416 ff. 
cvidence against, 351 


and hedonism-homeostasis Issue, 275+ 
305 


repudi 


ation of negative half, 412 

and retroaction, 361 

"E horndike's statement of, 23 

and two-factor learning theory, 2547 


E 
and unlearning, 389 


a useful prototype 


aw of etfects, 257 


aw of exercise, 410-411 
evidence against, 410 
and law of effect, 411 
reinterpreted, 244, 411 
Learning, and abstraction, 328 


32 


active avoidance, 27-28, 30731, 54 [E 
all-or-none, 483 

awareness of, 14 

basically emotional, 307 

biological utility of, 387 

capacity for, 9 

as conditioning, 19, 62, 419 

and consciousness, 13-14 

as à construct, 13 


continuous or discontinuous, 439 
criteri 


a of, 340 

with delayed reinforcement, 354. 387 
by doing, 18, 215 

without doing, 18, 215. 217. 228 

and drive reduction, 6^ 

the environment, 302 


essentially cognitive, 302 


of expectations, 325 


facilitated by punishment, 434 
and generalization, 438 

higher forms of, 329 

as induction, 438 
instrumental, 57 
Integrative, 378 
laws of, 9 
locus of, 397 


material substratum for, 219. 238 
mathematical models of, 222 
à matter of control, 221 


of meanings, not movements, 330 
and memory, 303 i 
nature of, 397 

neurological correlates of, 
always generalization, 43 


EJ 


7 ff... 23 
sff. 


as once 
passive, 215 

and past experience, 4 
and perception, 302 
and performance, 267 


88 
without reinforcement, 250-281 
attractiveness or aversion, 


positive and negative, 3 


as response 
220 


as response probability, 219 


reversible, 
as SOR bonding, 124. 329 
selective, 67 
no sensation of, 13 
77-78. 68, 


of signs and solutions, 65. 7777 
1 


3 
social, 8 
382 


supe rstitious, 
apolation, 438 


as tempo al extr; 
76 

two conceptions of, 
two-factor theory of, 


too slow, 
123 


19, 63 ff. 


types of, 
and unlearning, 39° 


Vicarious, 19 


and extinction, successive, 45! 


Learning 

Learning theory 
and behavior theory. 340 
and common sense, 47 
false security of, 289 
limitations of. 7 ff. 
and mental content, 1! 
neglect of cognition, 305 
three forms of, 277 
value of, 289 

rning to learn, 452 

p 11, 383 


earning sets 


Leg flexion, conditioned. 
i ET 


allv clicited 
s s of, 292 ff. 


clectri 
Konorskts studic 
ar, 238 
cd, 238 
f, 281 


motivated by fe 
passively produc 
preconditioning 
Leucotomy, 136 


for intractable 


of, 


pain, 137 
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Love, 392 


Mammals, newly born, 190 
Man, the symbol user, 379 
Manipulation, 178 
Masochism, 177. 436 
Mathematical models, 
9 

Meaning, 20, 39. 143+ 400 
Meanings, 509731! 
Means-end expectation 
Means-end relations, 321. 329 
Mediating processes, 309 
Mediation, and generalization, 442 


limitations 


by images, 282 
and language theory, 438 
emote reinforcement, 380-385 


ofr 
i ced stimuli, 327 


vid 


re ponse-produ 
Memory, 20, 252. 33+ 438 
Mind, functionally conceived, 2-4 
Miron paradox, 130 ff. 
Mislearning, 356. 377 
Models, 12-135 17 
of Hull and Spence, 11 
scientific, 1 ff., 12 
Money. 39 
Monotony, 186 
Morphine, effects of, 135 


196 ff. 273 


Motivation, and activity 
and appetite. 196 ff. 
and autonomic nervous sy stem, 203 

and behavior in space, 316 

and the brain, 211 

competing. 

criterion of, 272 

and depression, 197 


and emotion, 273. 435 


and habit, 173 
homeostatic v iew of, 182 
intrinsic, 174 

arning criterion of, 273 


le 
and motility. 269 
multiplicit of, 341 


nature of, t 204 
negative aspects of, 178 
and relief, 271 


secondary. 34 


547 


of, 
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Motivation € continued ) 
stimulus-intensity model, 


tertiary, 201 
visual, 179 
Motives (see also Drives) 
basically negative, 179 
competing, 338 
externally aroused, 177 
learned, 273 
and motions, 173 
not self-enhancing, 179 
primary and secondary, 273 
self-climinating, 179 
Motor learning, 488 
Motor pathways, not learned, 221 
Mourning, 425 
Movement, feedback from, 298 
Muscular contractions, sensations of, 258 


Natural variation, 17 
Nausea, 417 
conditioned, 270 
Need, 186 ff. 
reduction in, 260 
Needs, biological, 275 
Need-primacy theory, 300 
Negative adaptation, 395 
Negative practice, 410 
Neobchaviorism, 6 
Nervous system, autonomic, 40, 202 ff.. 
402, 407 
central, 205 
Neural circuits, 365 
Neural pathways, and learning, 217, 391 
(see also Bonds) 
Neural reverberation, 361, 364 ff. 
Neurological reductionism, dan 
219 
Neurology, distrust of, 222 
entanglements with, 218 
of learning. 219. 238 


gers of, 


Neurophysiology, trends in, 222 
Neurosis. 


9 
as discrimination failure, 487 
experimental, 40 
ice-box, 162 
Neutroceptors, and reality, 317 
Nociception, 316 


Nonadjustive behavior, 378 
Nonconfirmation, versus countercondi- 
tioning, 398 ff. 
Nonreinforcement, and. countercondi- 
tioning, 419, 422 
Nonreward, an active process, 409 
cue value of, 451 
its emotional effects, 410 


Obedience, 485 

Obedience training, 487 

Obligation, 8 

Olds-Milner effect, as addiction, 209 
and normal reward, 210 

Operant behavior, 426 

Operant conditioning, 380 
extinction of, 427 

Optimism, 79. 169 

Organism, as 


an open system, 304 
Orgasm, and brain stimulation, 207, 209 


Pain, anticipation of, 136 
as a barrier, 433 
as conditioned stimulus, 433 
from electrical stimulation, 239 
expectancy of, 136 
and fatigue, 199 
as a function of fear, 136 
intractable, 137 
Palatability, 188-191, 276, 395 
Panic, 1 T 
Parachute jumping, 139 
Partial irreversibility, 485, 487 
Partial reinforcement, 469 
Passion, 200 
Passive avoidance also Punish- 
ment) 
abrogation of, 435 
produced by punishment, 256, 389 
and response learning, 335 
Pathways, in the brain, 222 
Pattern of reinforcement, predictable 


and unpredictable, 475 
Perception, 211 


(see 


and emotion, 279 
Perceptual field, 313 
Perceptual reorganization, 282 
Performance, changes in, 265 

as criterion of learning, 340 
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determined by demands, not bonds, 
326 À 
and learning, 267, 323 
rapid, 490 ff. 
speed of, 493 
Perseverance, 435 ff. 
ditferently evaluated, 47! 
Persistence, 416 
Personality, 9-10. 13 
breakdown of, 379 
Personality development, 304 


Jer: 1 
Personality theory, H 


sm, 169 
optimal state of, 186 


Pessin 

Physiolog 

Place learning. 329 ff. 
and field behavior, 308 
and habit theory, 398 
and response learning. 


306, 334 
Placebo etfect, 135 

Play, 172, 318 

Pleasantness, 318 


Pleasure, 203, 276 
Pleasure centers, 209 
Pleasure principle, 436 
Position habit, 413 
Postremity, principle of, 284 
Preconditioning, 235+ 280-281 
and images, 282 
Predictability, 34! 
Prediction, of constancy 
in open ficld, 338 
limits of, 304 
of movements, 284 


or change. 284 


Pre-extinction, 235 
Preference, and res 
461 
Primary drives 
reinforcing but n 
Primary reinforcers, 
Primary reinforcement, 
mote, 384 
two forms of, 103 


istance tO extinction, 


, not motives, 273 
ot motivating, 274 
value of, 45! 


temporally re- 


cuc 


Privation, 203 
Probabilistic learning. 
Problem situation, 1 
Problem solving. and 
8 
ation, 173 


461 
soluble. 4!? 
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